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Section  1.0 
Introduction 


1.1  Background 

Under  separate  contract  to  the  Army  Research  Institute  Aviation  Research  and  Development 
Activity  (ARIARDA)  Dynamics  Research  Corporation  (DRC)  developed  three  measures 
(instruments)  of  crew  coordination;  one  attitude-based  measure  and  two  performance-based 
measures. 

0  The  attitude-based  measure  is  the  Army  Coclq)it  Management  Attitudes  Questionnaire 
(Army  CMAQ).  The  Army  CMAQ  is  a  questionnaire  asking  aviators  to  rate  the  extent 
of  their  agreement/disagreement  to  45  statements  regarding  their  attitudes  towards 
aircrew  coordination.  Agreement/disagreement  is  recorded  using  a  7-point  Likert  scale. 
The  Army  CMAQ  can  be  found  at  Appendix  A.l, 

0  One  of  the  performance-based  measures  of  behavior  is  the  Aircrew  Coordination 
Evaluation  Checklist  (ACE  Checklist).  The  ACE  Checklist  is  filled  out  by  an  observer/ 
evaluator,  typically  an  Instructor  Pilot  (IP).  The  ACE  Checklist  consists  of  19  aircrew 
coordination-related  behaviors  with  each  behavior  rated  on  a  7-point  scale  ranging  from 
Very  Poor  to  Superior.  All  19  behaviors  are  described  and  "behaviorally  anchored"  at 
the  1,  4,  and  7  level.  The  ACE  Checklist  (without  the  behavioral  anchors)  can  be  found 
at  Appendix  A.2. 

0  The  other  performance-based  measure  of  behavior  is  based  on  revisions  to  the  tasks 
defined  in  TC  1-212.  Aircrew  Training  Manual  Utility  Helicopter.  UH-6Q  (hereinafter 
referred  to  as  ATM  Tasks).  Revisions  were  made  for  the  purpose  of  including  aircrew 
coordination  considerations  into  selected  ATM  Tasks.  ATM  Task  performance  was  rated 
as  an  A,  B,  C,  or  U  using  a  modified  standard  form  gradeslip.  The  modified  gradeslip 
used  in  this  study  can  be  found  at  Appendix  A. 3. 

A  complete  description  of  the  development  of  the  measures  and  all  supporting  documentation 
is  contained  in  Technical  Report:  Development  of  Measures  of  Crew  Coordination  dated  31 
August  1990  (hereinafter  referred  to  as  the  Development  of  Measures  technical  report). 

In  this  report,  DRC  provides  the  results  of  a  data  analysis  that  examined  the  functional 
relationships  between  aircrew  coordination  attitudes,  aircrew  coordination  behaviors,  and  mission 
performance.  Under  the  sponsorship  of  ARIARDA,  DRC  collected  the  data  for  this  analysis  at 
Fort  Campbell  during  the  Spring  of  1990  and  at  Fort  Rucker  during  the  Summer  of  1990. 

The  remainder  of  this  Section  reports  on  the  methodology  and  sample  used  for  the  data 
collection.  Sections  2  through  5  report  on  the  internal  properties  of  the  measures  used  in  the 
study.  Sections  6  through  9  present  a  variety  of  correlation-type  analyses  used  to  determine  the 
relationships  among  the  measures.  Section  10  contains  a  summary  and  conclusions  based  on  the 
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previous  Sections;  potential  additional  studies  are  suggested  which  may  be  of  interest  to  the 
Army. 

1,2  Methodology  and  Sample  Description 

As  depicted  in  Table  1.2-1,  there  were  several  administrations  of  the  three  aircrew  coordination 
(AC)  measures.  Additionally,  one  other  instrument  was  used  as  part  of  a  broader  admimstration 
of  the  Army  CMAQ  at  Fort  Campbell  on  30  May  1990.  At  that  time,  unit  Instructor  Pilots  (IP) 
were  asked  to  provide  "quality  ratings"  on  aviators  within  their  unit  using  a  predefmed, 
standardized  measurement  form  and  scale.  This  form  is  at  Appendix  A.4. 

Data  collection  used  the  following  organizations  and  personnel: 

0  Testbed  Aviators  -  Forty  aviators  comprising  twenty  crews  from  the  101st  Aviation 
Regiment  participated  in  the  May  90  testbed.  The  twenty  crews  were  given  an  identical 
two-hour  tactical  mission  to  fly  in  the  UH-60  flight  simulator.  All  forty  aviators 
completed  the  Army  CMAQ;  the  twenty  crews  were  rated  on  the  ACE  Checklist  and 
ATM  Tasks. 

0  Testbed  IPs  and  I/Os  -  Three  IPs,  serving  as  raters  of  aircrew  performance,  and  four 
simulator  Instructor/Operators  (I/O),  serving  as  simulator  operators,  participated  in  the 
testbed.  These  seven  individuals  were  given  familiarization  training  in  the  principles  and 
practice  of  aircrew  coordination;  then  fully  trained  to  implement  the  testbed  simulation 
procedures  and  the  rating  instruments.  As  part  of  their  familiarization  training  in  aircrew 
coordination,  the  IPs  and  I/Os  were  administered  the  Army  CMAQ  before  and  after 
training. 

o  101st  Avn  Regiment  -  Subsequent  to  the  testbed,  eighty  (80)  aviators  from  the  4th,  5th 
and  9th  Aviation  Battalions  of  the  101st  Aviation  Regiment  were  administered  the  Army 
CMAQ  on  30  May  90.  Of  those  80  aviators  completing  the  Army  CMAQ,  58  aviators 
received  "quality"  ratings  from  their  unit  IP. 

0  USASC  -  United  States  Army  Safety  Center  (USASC)  personnel  were  trained  during 
June-July  90  in  methods  to  incoiporate  aircrew  coordination  considerations  into  accident 
investigations.  DRC  fully  trained  20  USASC  personnel;  an  additional  two  USASC 
personnel  were  able  to  only  partially  complete  the  training.  Twenty  (20)  USASC 
personnel  were  administered  the  Army  CMAQ  prior  to  training;  of  those  14  also 
completed  it  subsequent  to  training. 

Different  sample  sizes  are  used  in  the  analyses.  To  decode  the  meaning,  each  is  defined  below; 

n=20:  the  twenty  testbed  aircrews. 

n=40:  the  forty  testbed  aviators. 

n=80:  Fort  Campbell  aviators  taking  the  Army  CMAQ  on  30  May  90. 
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n=58:  Fort  Campbell  aviators  taking  the  Army  CMAQ  on  30  May  90  who  also 

received  "quality"  ratings  from  their  unit  IP. 
n=168:  all  those  who  took  the  Army  CMAQ  under  all  conditions/places. 

There  is,  of  course,  some  missing  data.  On  these  occasions,  a  particular  equation  or  result  within 
a  Table  is  based  on  slightly  fewer  subjects  than  the  "n"  noted  in  the  Table. 


Table  1.2-1 
Administration  of  the 
Aircrew  Coordination  Measures 

Army  ATM  Quality 


Sample 

Location 

CMAO  ACE 

Tasks 

Ratines 

Testbed 

aviators 

Ft. 

Campbell 

X  X 

(n=40)  (n=20) 

X 

(n=20) 

Testbed 

IPs  &  I/Os 

Ft. 

Campbell 

Pre-  St 

Post 

(n=7) 

101st  Avn 
Regiment 

Ft. 

Campbell 

X 

(n=80) 

X 

(n=58) 

USASC 

Ft. 

Rucker 

Pre  (n=20) 

Post  (n=14) 

Total 

All 

n=168  n=20 

n=20 

n=58 
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Section  2.0 

Properties  of  the  Army  CMAO 


2.1  General 

Scales  and  scale  scores  were  created  for  the  Army  CMAQ.  The  Army  CMAQ  subscales  were 
slightly  modified  from  the  initial  scales,  but  were  still  based  on  the  conceptual  framework 
presented  in  the  Development  of  Measures  technical  report.  The  data  collected  at  Ft.  Campbell 
and  USASC  were  based  on  the  first  administration  of  the  Army  CMAQ,  the  initial  version  of 
which  can  be  found  in  Appendix  A.l.  Frequency  distributions  were  computed  for  each  CMAQ 
item;  and  item  analyses  were  performed  for  each  of  the  derived  CMAQ  subscales.  Frequency 
distributions  and  sc^e  construction  are  further  discussed  below. 

2.2  Frequency  Distribution 

Appendix  B  contains  frequency  distributions  for  all  45  Army  CMAQ  items.  [Note:  the  45 
CMAQ  items  are  referred  to  sequentially  as  Cl  to  C45.]  Respondents  availed  themselves  of  the 
entire  seven  point  rating  scale;  therefore,  most  items  have  a  reasonable  amount  of  variability 
associated  with  them. 

2.3  Scales  and  Scale  Construction 

During  the  initial  development  of  the  Army  CMAQ,  DRC  constructed  five  subscales  comprising 
all  45  items  of  the  Army  CMAQ.  During  Ae  current  research  phase,  the  scales  were  refined  and 
reorganized  into  four  subscales  as  summarized  below: 

1)  The  attitude  previously  summarized  as  "Values  Crew"  was  redefined  as  "Values  Teamwork. " 
In  accordance  with  this  change.  Table  2.3-1  describes  the  revised  linkages  between  between 
beliefs,  attitudes,  and  behaviors  (Table  2.3-1  may  be  compared  to  Table  2.3-5  of  the 
Development  of  Measures  technical  report).  Note  that  the  third  column  labeled,  "essential 
crew  attitudes,"  is  the  column  that  has  been  altered. 

2)  The  two  attitude  areas  previously  summarized  as  "Get  Information"  and  "Give  Information" 
were  combined  for  two  reasons:  one  reason  being  that  combining  "give"  and  "get"  made  the 
attitude  area  better  aligned  with  the  "Provide/ Accept  Help"  attitude  area,  i.e.,  sharing 
information,  like  helping,  is  now  a  "two-way  street;"  the  second  reason  was  that  combining 
the  two  subscales  increased  the  number  of  items  appearing  in  the  subscale,  thus  improving 
reliability. 

3)  The  items  were  uniquely  placed  into  each  subscale.  While  it  was  conceptually  possible  that 
items  could  fall  into  more  than  one  attitude  area,  consideration  had  to  be  given  to  the 
ramifications  for  subsequent  data  analyses;  e.g.,  correlations  and  regressions.  If  items  were 
allowed  to  appear  in  more  than  one  subscale,  then  the  subscales  would  be  dependent  upon 
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one  another  and  the  results  would  be  difficult  to  interpret.  Using  Table  2.3-1  for  guidance,  DRC 
analysts  uniquely  placed  the  Army  CMAQ  items  into  distinct  "logical"  subscales  based  on 
subjective  judgements  concerning  the  attitude  area  best  matched  by  the  item.  In  accordance  with 
this  change,  Table  2.3-2  presents  the  new  organization  of  the  Army  CMAQ  items  into  the 
"logical"  subscales  and  updates  the  subscale  organization  previously  defmed  in  Table  5.1-1  of 
the  Dp-velonment  of  Measures  technical  rqxirt. 


Tabl*  2.3-2 

Placuien't  of  Army  CKAQ  Z'taas  int.o  Subscales 


Army  CMAQ  Subscale/ 

At-titude  Area  Army  CMAQ  Item  Number 


Values  Teamwork 

1, 

4,  ! 

5/  7, 

-  8, 

9, 

15, 

19, 

22, 

25, 

26, 

27, 

r  29, 

,  30, 

.  42, 

/  44 

,  45 

Crew  Fallibility 

6, 

11/ 

12, 

14, 

17, 

18, 

20, 

21, 

28, 

38, 

41 

Give/Get  Information 

2, 

10, 

13, 

23, 

24, 

31, 

32, 

33, 

34, 

43 

Provide/Accept  Help 

3, 

16, 

35, 

36, 

37, 

39, 

40 

Total  1  to  45 


2.4  Army  CMAQ  Reliability  Statistics 

Respondents  answered  the  45  Army  CMAQ  items  using  a  7-point  scale  ranging  from  Strongly 
Disagree  (value  =  1)  to  Strongly  Agree  (value  =  7).  For  24  of  the  45  items,  the  desirable,  or 
"correct,"  attitude  was  a  value  of  7.  The  remaining  21  items  were  negatively  worded  so  that  a 
value  of  1  was  the  desirable  or  "correct"  attitude.  Before  proceeding  with  the  analyses,  all  items 
were  "scored."  The  scoring  key  is  provided  at  Table  2.4-1. 
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Table  2.4-1 

Item  by  Item  Scoring  Key  Employed 
with  Army  CMAQ  Items 

Item  Numbers  for  Item  Numbers  for 

»ArfT-P,P-»  =  Correct  "Disagree'*  =  Correct 

2,  3,  4,  5,  6,  7,  8, 

10,  13,  15,  16,  17, 

18,  19,  23,  25,  28,  30 
31,  32,  33,  36,  37,  44 


After  "scoring"  the  responses  to  the  items,  several  types  of  reliability  analyses  were  performed 
on  each  of  the  four  subscales  and  the  total  score.  The  reliability  statistics  calculated  for  the 
Army  CMAQ  include  Cronbach’s  Alpha,  Split-Half,  and  Test-Retest  reliability  statistics  (Table 
2.4-2),  Cronbach’s  Alpha  and  Split-Half  reliabilities  were  calculated  based  on  the  entire  sample 
of  subjects  administered  the  Army  CMAQ  (n=168);  one  case,  however,  was  dropped  due  to 
missing  data.  Test-retest  reliability  was  calculated  based  on  those  subjects  administered  the  Army 
CMAQ  on  two  different  occasions  (n=35).  This  group  included  testbed  aviators  who  participated 
in  the  May  90  Army  CMAQ  administration,  testbed  IPs  and  I/Os,  and  USASC  personnel. 


1,  9,  11,  12,  14,  20,  21,  22 
24,  26,  27,  29,  34,  35,  38,  39, 
40,  41,  42,  43,  45 


Table  2.4-2 

Comparative  Scale  Reliability  Statistics 
for  the  Army  CMAQ 
(n=l68) 


#  of 

Avg . Item 

Scale  1 

Reliabilities** 

Scale  Name 

Items 

Mean 

S.D. 

1 

2 

3 

4 

1)  Total 

45 

5.47 

.38 

.78 

.66 

.75 

++ 

2)  Values  Teamwork 

17 

5.26 

.45 

.51 

.33 

.57 

.42 

3)  Crew  Fallibility 

11 

5.23 

.58 

.52 

.44 

.59 

.  50 

4)  Give/Get  Information 

10 

5.83 

.52 

.  64 

.  48 

.81 

.  67 

5)  Provide/ Accept  Help 

**  Reliabilities: 

7 

5.83 

.51 

.55 

.44 

.40 

.61 

1  =  Cronbach's  Alpha  (n=167) 

2  =  Split-Half  (n=167) 

3  =  Test-Retest  (n=35) 

4  =  Cronbach's  Alpha  (n=40,  testbed  aviators) 

++  Determinant  of  matrix  is  zero;  could  not  be  computed. 


As  Table  2.4-2  shows,  the  reliability  coefficients  of  the  Army  CMAQ  and  the  subscales  are  good 
and  are  similar  to  those  reported  by  the  NASA/UT  Crew  Performance  Project  (Gregorich  et  al., 
1990).  The  NASA/UT  project’s  three  subscales  were  derived  based  on  a  factor  analysis. 
Cronbach  Alphas  were  reported  for  CMAQs  given  to  personnel  from  three  commercial  airlines, 
the  third  of  which  had  the  CMAQ  administered  in  a  pre-  and  post-training  condition  (Table  2.4- 
3).  The  NASA/UT  subscale  reliabilities  range  from  .46  to  .67;  DRC/ARIARDA’s  subscale 
reliabilities  range  from  .51  to  .64  for  a  much  smaller  sample. 


Table  2.4-3 

Cronbach 's  Alphas  for  a  Three  Siibscale  CMAQ 
as  Reported  by  MASA/CT 
(Gregorich  et  al.,  1990) 


Scale  Name 

Airline  A 
fn=3741 

Airline  B 
fn=3774) 

Airline  C 
Pre-test 
/n=696) 

Airline  C 
Post-test 
fn=701) 

Communication  & 
Coordination 
(11  items) 

.57 

.67 

.63 

.67 

Command 

Responsibility 
( 4  items ) 

.52 

.46 

.48 

.47 

Recognition  of 

.60 

.52 

.59 

.60 

Stressor  Effects 
( 4  items ) 


A  correlation  matrix  (Table  2.4-4)  was  generated  to  show  the  relationship  among  the  Army 
CMAQ  scales. 
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Table  2.4-4 

Army  CMAQ  Sxibscale  Correlations 
(n=168) 


Scale  Name 

Total 

Values 

Teamwork 

Crew 

Fallibility 

Give/Get  Provide/ 
Info  Acceot  Helo 

Total 

— 

Values  Teamwork 

.82 

— 

Crew  Fallibility 

.74 

.45 

— 

Give/Get  Info 

.77 

•47 

.40 

— 

Prvde/Accpt  Help 

.60 

.32 

.24 

.53  - 

A  comparison  of  the  subscale  correlations  to  the  NASA/UT  inter-composite  correlations  showed 
the  Army  CMAQ  correlations  to  be  higher.  The  NASA/UT  correlations  ranged  from  .00  to  .27 
(correlations  were  not  provided  for  the  Total  score  with  the  subscale  factors).  Note  that  the 
correlations  of  the  Total  column  in  Table  2.4-4  with  the  subscales  are  high  since  subscale  items 
are  embedded  within  the  Total  score.  It  is  desirable  when  constructing  subscales  that  the 
relationships  among  them  be  low  since  that  implies  the  subscales  are  assessing  different 
attributes.  In  the  case  of  the  Army  CMAQ  data  the  subscale  correlations  are  relatively  low,  but 
not  as  low  as  those  the  NASA/UT  project  showed  from  data  collected  in  the  commercial  aviation 
sector.  It  may  be  that  the  higher  inter-scale  correlations  of  the  Army  CMAQ  are  due  to  the 
manner  in  which  Army  aviators  understand  aircrew  coordination;  Army  aviators  may  view 
aircrew  coordination  as  a  more  integrated  concept  with  the  subscales  being  more  mutually 
supportive  of  one  another. 

2A  Factor  Analysis  of  the  Army  CMAQ 

A  factor  analysis  was  performed  on  the  Army  CMAQ  data  to  determine  if  alternative,  and 
perhaps  more  meaningful  scales  could  be  developed.  To  capture  the  underlying  factors  as 
presented  by  "field"  Army  aviators,  the  sample  used  in  the  factor  analysis  was  limited  to  Army 
unit  aviators:  i.e.,  the  eighty  (n=80)  Ft.  Campbell  aviators  administered  the  Army  CMAQ  in 
May  1990. 

When  the  45  Army  CMAQ  items  were  factor  analyzed  without  constraints,  a  15-factor  model 
resulted.  Since  a  15-factor  model  proved  unwieldy,  the  data  were  alternatively  limited  to  four, 
and  then  three  factors.  The  four  factor  model  was  not  readily  interpretable;  the  items  did  not 
collect  in  an  explicable  manner.  The  three-factor  model  proved  most  interpretable.  The  rotated 
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(varimax)  factor  matrix  converged  in  9  iterations  with  30.7%  of  the  variance  explained.  Table 
2.5-1  shows  how  the  items  loaded  on  each  factor.  Note  that  the  items  in  Table  2.5-1  are  labeled 
Cl  through  C45.  These  labels  correspond  to  the  45  item  numbers  in  the  Army  CMAQ  found  in 
Appendix  A.  1 . 

The  three  factors  shown  in  the  Table  2.5-1  are  similar  to  the  ones  discussed  by  NASA/UT 
(Gregorich,  et  al,  1990).  In  that  article,  the  authors  named  the  factors  "Communication  and 
Coordination",  "Command  Responsibility",  and  "Recognition  of  Stressor  Effects." 

Since  the  factor  analysis  of  the  Army  CMAQ  resulted  in  "reasonable"  factors  and  because  the 
Army  factors  closely  approximated  previous  research,  it  was  decided  to  further  explore  the 
"factor"  scales  in  addition  to  the  "logical"  scales  discussed  in  previous  sections.  Since  the  Army 
CMAQ  contains  45  items,  and  the  NASA/UT  CMAQ  contains  25  items  (19  of  which  were 
"consistently  identified"  across  the  four  samples  as  loading  on  a  particular  factor),  an  approach 
for  categorizing  the  Army  CMAQ  items  into  the  three  factor  scales  was  developed  based  on 
three  decision  rules: 

1)  Army  CMAQ  items  1  through  21  were  included  in  a  scale  if  they  loaded  similar  to  the 
NASA/UT  CMAQ  items, 

2)  Army  CMAQ  items  22  through  45  were  placed  into  the  scale  on  which  their  highest 
loading  occurred;  and 

3)  Negative  highest  loadings  (specifically  items  25  and  29)  were  excluded  from  the  scales. 

Placement  of  the  34  Army  CMAQ  items  meeting  the  above  criteria  into  the  three  factors  is 
shown  in  Table  2.5-1. 


)  _ _ _ _ 

)  - - - - 

Table  2-5.1 

i  Factor  Analysis  Results  for  the  Army  CMAQ 

i  (n=80) 


CMAQ 
Item  #+ 

FACTOR  1 
rcomm  &  Coord) 

FACTOR  2 
f Shared  Ldrshn) 

FACTOR  3 
( Stressors) 

Cl 

. 04600 

.29676 

.36115 

C2 

.41243* 

.28932 

-.22092 

C3 

.28068 

.09211 

-.28248 

C4 

-.12962 

.40165 

-.13770 

C5 

. 19132 

-.26749 

.54709 

C6 

.58756* 

.20298 

.00737 

C7 

.57549* 

.31658 

.12830 

C8 

.51250* 

-.01470 

.38057 

C9 

-.49295 

.20670 

.25609 

CIO 

.27342 

.34128 

-.09177 

Cll 

-.34842 

.03082 

.43091 

C12 

-.01262 

.03260 

.41666* 

C13 

.21929* 

.09648 

.04967 

C14 

-.21957 

.59598* 

.15562 

C15 

. 65339* 

.26947 

-.00042 

C16 

.53773 

-.07450 

.11731 

C17 

.01410 

-.13229 

-.39910 

C18 

. 61427* 

-.03345 

-.04611 

C19 

.61589* 

.07438 

-.02910 

C20 

.11052 

.07188 

.60121* 

C21 

-.17113 

. 01174 

.48509* 

C22 

-.41596 

.34817 

.50407* 

C23 

.52093* 

.48507 

-.18701 

C24 

.26311 

.38707* 

.31534 

C25 

-.18785 

-.24067 

.20237 

C26 

.01557 

.37260* 

.09224 

C27 

.36661 

.72399* 

.04169 

C28 

.26037* 

-.11576 

.06070 

C29 

-.36824 

.19315 

.29994 

C30 

.25185* 

.08104 

.04709 

C31 

.53099* 

. 16719 

-.03701 

C32 

.18946 

. 33896* 

-.01618 

C33 

.52965* 

.40418 

-.25547 

C34 

.11832 

.35523 

.48046* 

C35 

. 17714 

. 59047* 

-.00225 

C36 

.65711* 

. 31763 

.08394 

C37 

.49190* 

.13966 

-.19401 

C38 

.20497 

.42666* 

.24625 

C39 

.05080 

.48586* 

.15605 

C40 

.29671* 

.23765 

.17983 

C41 

.13690 

.36576 

.47943* 

C42 

.05746 

.53636* 

.01603 

C43 

.04969 

.32444* 

.15793 

C44 

.30252* 

-.17383 

.26656 

C45 

.06307 

.44219* 

.29971 

*  Item 
+  CMAQ 

included  in  this  "factor”  scale. 

Item  #  Cl  to  C45  are  CMAQ  item  numbers  1 

to  45 . 
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Note  that  Factor  2  has  been  renamed  from  the  NASA/UT  nomenclature  of  "Command 
Responsibility”  to  "Shared  Leadership"  to  provide  a  better  description  of  the  factor.  To 
summarize,  34  Army  CMAQ  items  were  selected  and  placed  into  three  "factor"  scales  as  a  result 
of  a  factor  analysis  and  several  decision  rules.  The  resulting  Army  CMAQ  "factor"  scales  are 
shown  in  Table  2.5-2. 


Table  2.5-2 

Placement  of  Army  CMAQ  Items 
into  the  "Factor''  Scales 


Factor  #  Scale  Name 


1  Communication  & 

Coordination 


2  Shared  Leadership 


Army  CMAQ  Items  in  Scale 


C2,  C6,  Cl,  C8,  C13, 
CIS,  C18,  C19,  C23, 
C28,  C30,  C31,  C33, 
C36,  C37,  C40,  C44 


C14,  C24,  C26,C27,  C32, 
C35,  C38,  C39,  C42,  C43 
C45 


3  Recognition  of 

Stressor  Effects 


C12,  C20,  C21,  C22, 
C34,  C41 


Overall  (34  "selected"  items) 


All  above  listed  items. 


Factor  definitions: 

1)  ConunuiiicEtioii  &  Coordination  -  an  orientation  toward  interpersonal  awareness, 
communication,  and  crew  coordination.  Example  items  are: 

"Crewmembers  should  feel  obligated  to  mention  their  own  personal  psychological  stress  or 
physical  problems  to  other  crewmembers  before  and  during  a  mission. 

"The  pilot-in-command’s  responsibilities  include  coordination  of  inflight  crewchief 
responsibilities. " 

"The  pilot-in-command  should  use  his  crew  to  help  him  maintain  situation  awareness." 

2)  Shared  Leadership  -  an  attitude  toward  the  appropriateness  of  sharing  responsibility  for 
leadership.  Example  items  are; 
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•When  joining  a  unit,  a  new  crewmember  should  not  offer  suggestions  or  opinions  unless 

asked."  (negative  response) 

"Pilots-in-command  who  accept  and  implement  suggestions  from  the  crew  are  lessening  their 
stature  and  reducing  their  authority."  (negative  response) 

"The  pilot-in-command  should  seek  advice  from  crewmembers  in  updating  mission  plans." 

3)  Recognition  of  Stressor  Effects  -  an  attitude  accepting  that  human  performance  is  affected 
by  external  events  and  allowance  must  be  made  for  changed  performance.  Example  items  are. 

"Even  when  fatigued,  I  perform  effectively  during  most  critical  flight  maneuvers."  (negative 
response) 

"Most  crewmembers  can  leave  personal  problems  behind  when  flying  a  mission,  (negative 
response) 

"My  decision  making  is  as  good  in  emergencies  as  in  routine  situations."  (negative  response) 

Upon  inspection,  linkages  were  developed  between  the  "logical"  (Table  2.3-1)  and  the  factor 
scales  (Table  2.5-3).  As  wiU  be  seen  in  Sections  6-9,  there  is  also  some  empirical  evidence  that 
justifies  the  relationships  depicted  in  Table  2.5-3. 


Table  2.5-3 

Linkage  between  the  Army  CMAQ 
••Logical”  and  ••Factor”  Scales 


"Logical”  Scale 
Values  Teamwork 
Crew  Fallibility 

Give/ Get  Information 
Provide/ Accept  Help 


” equals"  "Factor"  Scale 

=  Shared  Leadership 

=  Recognition  of  Stressor 

Effects 

=  Communication  &  Coordination 

=  Communication  &  Coordination 


After  developing  the  "factor"  scales,  scale  and  subscale  statistics  and  reliability  coefficients  were 
computed.  The  results  are  presented  in  Table  2.5-4.  Of  note  is  that  the  '  factor  scales  for  the 
Army  CMAQ  yielded  higher  reliability  coefficients  than  both  the  NASA/UT  CMAQ,  which 
utilized  a  much  larger  sample  but  fewer  items  for  each  of  the  subscales,  and  the  45-item  Army 
CMAQ  "logical"  scales. 
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Table  2.5-4 

comparative  Scale  Reliability  statistics 
for  the  Army  CMAQ  Using  "Factor  Scales 

(n=80) 


of 

Avg . Item 

Scale 

Reliabilities  ** 

eitis 

Mean 

S.D. 

1 

2 

17 

5.90 

.45 

.77 

.77 

•  61 

11 

5.69 

.61 

.75 

.80 

.  54 

6 

4.43 

.96 

.67 

.69 

.49 

34 

5.57 

.43 

.81 

.85 

•  68 

Sr.ale  Name 

1)  Comm.  &  Coor . 

2 )  Shared  Leadership 

3)  Stressor  Effects 

4)  Total  (34  Items) 

**  Reliabilities: 

1  =  cronbach's  Alpha  (n-80)  a-i-nrs^ 

2  =  cronbach's  Alpha  (n=40,  testbed  aviators) 

3  =  Split-Half  (odd-even)  ^n=80) 


1  mofriv  rTnhie  1  S-5')  was  generated  to  show  the  relationship  among  the 
"fa^oJ"  sc^^^Note  that  the  "factor"  subscale  intercorrelations  are  generally  lower  than  those 
of  the  "logical"  subscale  correlations  shown  in  Table  2.4-4. 


Table  2.5-5 

Army  CMAQ  "Factor"  Subscale  Correlations 

(n=80) 


■Snale  Name 
Total  (34  items) 
Comm.  &  Coor. 
Shared  Leadership 
Stressor  Effects 


Total 

.72 

.82 

.60 


Comm. 

Coor. 


.37 

.05 


Shared 

T.eadership 


.38 


Stressor 

Effects 


2.6  r.nmparison  of  Army  and  Civilian  CMAQ  Response! 

In  nn  article  discussing  the  relationship  between  attitudes  ^d  f 

Performance  Project  (Helmreich  et  al.,  1986)  desmW  the  °f 

Check  Airmen)  commercial  aviators  to  selected  CMAQ  items.  NASA/UT  found  that,  m  genem, 

superior  aviators  tend  to  hold  similar  attitudes  regarding  cockpit  resource  managemen^m 

aircrew  coordination.  DRC/AWARDA  was  also  interested  m  J 

in  the  Fort  Campbell  sample  were  similar  or  dissimilar  to  the  commerci  _i„tors  That 

NASA/UT,  and  2)  if  the  CMAQ  items  differentiated  between  good  and  poor 
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is  to  say,  in  relation  to  the  second  question,  could  a  high  or  low  quality  Army  aviator  be 
described  by  attitude  scores  as  obtain^  through  the  Army  CMAQ. 

To  determine  if  such  a  description  was  possible.  Army  aviators  were  separated  into  high  and  low 
performance  groups  based  on  their  scores  on  either  of  two  measures: 

o  Testbed  aviators  were  placed  into  high  or  low  groups  based  on  their  overall  performance 
on  the  ATM  tasks  (high  or  low  "performers"), 

0  Aviators  participating  in  the  mass  administration  of  the  Army  CMAQ  in  May  1990  were 
placed  into  high  or  low  groups  based  on  their  "quaHty"  ratings  assigned  by  the  umt  IP  (high 

or  low  "quality"). 


Tables  2.6-1  and  2.6-2  are  frequency  tables  showing  how  Army  aviators  scored  with  respect  to 
the  two  above  mentioned  measures.  After  inspecting  the  frequency  tables,  the  avmtors  w^e 
divided  into  high  and  low  ’’performance’*  or  ’’quality”  groups  as  indicated  in  the  Tables.  Ine 
"quality"  ratings  given  to  the  May  1990  group  were  arrived  at  through  the  use  of  the 
"Experimental  Ratings  of  Aviator  Qualities"  form  (Appendix  A.4).  For  ease  of  understanding 
and  to  make  desired  correlations  positive,  the  "quahty"  ratings  were  recoded  so  that  a  ° 
3  was  high  and  a  rating  of  1  was  low.  Of  the  80  Fort  Campbell  aviators  participating  m  the  May 
1990  administration  of  the  Army  CMAQ,  58  received  "quality"  ratings.  ReliabiUty  (Cronbach’s 
Alpha)  of  the  quality  ratings  was  computed  and  determined  to  be  .82. 
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Table  2.6-1 

Mean  Scale  Scores  on  ATM  Task  Performance 
and  Division  of  Testbed  Aviators 
into  High  and  Low  Performing  Groups 

(n=40) 


Scale 

Score  Frequency  Percent 


1.62 

1.69 

1.72 

2.04 

2.15 

2 

2 

2 

2 

2 

5” 

5 

5 

5 

5 

Low 

1 

1 

j 

••performers 

1 

1 

1 

.  1 

2.21 

2 

5“ 

2.22 

2 

5 

2.25 

2 

5 

2.28 

2 

5 

2.30 

2 

5 

2.38 

2 

5 

2.43 

2 

5 

2.46 

2 

5 

2.52 

2 

5 

2.69 

2 

5 

2.77 

2 

5“ 

“  1 

1 

2.91 

2 

5 

1 

1 

2.96 

2 

5 

High 

••performers 

3.00 

2 

5 

1 

3 . 08  2 _ 5 


Table  2.6-2 

Mean  Scores  on  "Quality”  Ratings  and 
Division  of  May  90  Group 
into  High  and  Low  Quality  Groups 

(n=58) 


Mean 

Score  Frecmencv  Percent 


1.00 

1.25 

1.50 

1.75 

3 

5 

6 

7 

5 

9 

10 

12 

1 

1 

Low 

I 

1 

1 

"quality" 

2.00 

14 

24 

2.25 

10 

17 

2.50 

4 

7 

! 

2.75 

3 

5 

High 

"quality" 

3.00 

6 

10 

TOTAL 

58 

Table  2.6-3  shows  a  CMAQ  statement,  the  direction  of  agreement  or  disagreement  of  the 
commercial  aviators,  the  mean  item  response  for  the  entire  sample  taking  the  Army  CMAQ, 
means  for  high  and  low  "performing"  testbed  aviators,  and  means  for  high  and  low  "quality" 
rated  May  90  aviators.  The  Table  covers  only  the  13  CMAQ  items  common  to  both  the 
NASA/UT  and  the  DRC/ARIARDA  studies.  Significant  differences  in  means  are  noted  with 
asterisks.  Differences  in  direction,  i.e.,  agree  vs.  disagree  between  commercial  and  Army 
responses  are  noted  with  plus  signs. 

Item  numbers  in  Table  2.6-3  are  ordered  in  the  manner  presented  in  the  Helmreich  et  al.  (1986) 
article  and  do  not  correspond  to  Army  CMAQ  item  numbering.  The  actual  item  numbers  from 
the  Army  CMAQ  are  C21,  C13,  C8,  C9,  C14,  Cll,  CIO,  C2,  C42,  C43,  C16,  C44,  and  C19, 
translated  into  item  numbers  1  through  13  respectively.  After  each  item  is  a  C##  indicating  the 
item  number  as  it  appeared  on  the  Army  CMAQ  (NASA/UT  CMAQ  items  10  and  15  have  no 
corresponding  Army  CMAQ  statement). 
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Tabl«  2.6-3 

Comparison  of  High  and  Low  Performer  Aviators 
on  Selected  CMAQ  Items 


Question: 

High 

ATM 

(n=10) 

Low 

ATM 

(n=10) 

High 

Qlty 

(nrl3) 

Low 

Qlty 

<n=21) 

1.  My  decision-makinfl  ability  is  as  good 
in  emergencies  as  in  routine  mission 
situations.  (C21)+ 

Disagree 

4.54 

5.37 

4.75 

5.38 

4.67 

2.  Pi  Lots- in- command  should  encourage 
pilots  and  crew  chiefs  to  question 
procedures  and  flight  profile  deviations 
during  normal  flight  operations  and  in 
emerqencies.  (C13) 

Agree 

5.44 

5.62 

6.00 

5.05 

3.  Crewmembers  should  be  aware  of  and 
sensitive  to  the  personal  problems  of 
other  crewmembers.  (C8) 

Agree 

5.76 

6.00 

5.62 

6.14 

5.71 

4.  The  pi  lot -in- command,  time  and 
situation  permitting,  should  take  control 
and  fly  the  aircraft  in  all  emergency  and 
non-standard  situations.  <C9) 

Agree 

3.70 

3.88 

3.12 

4.08 

3.71 

5.  There  are  no  circumstances  where  the 
pilot  should  take  the  aircraft  controls 
without  being  directed  to  do  so  by  the 
oi lot- in- command.  (C14) 

Disagree 

2.35 

2.50 

2.50 

3.54 

**  2.28 

6.  Pilots  and  other  crewmembers  should 
not  question  the  decisions  and  actions  of 
the  pi  lot- in- command  except  where  these 
actions  obviously  threaten  the  safety  of 
the  flight  (C11) 

Disagree 

3.17 

3.75 

3.00 

4.00 

3.05 

7.  The  pilot  flying  the  aircraft  should 
verbalize  plans  for  procedures  or 
maneuvers  and  should  be  sure  that  the 
information  is  understood  and 
acknowledged  by  crewmembers  affected, 

(CIO) 

Agree 

6.11 

6.25 

6,25 

6.23 

6.00 

8.  Crewmembers  should  feel  obligated  to 
mention  their  own  psychological  stress  or 
physical  problems  to  other  crewmembers 
before  or  during  a  mission.  (C2) 

Agree 

5.74 

5.62 

5.12 

5.77 

5.14 

9.  Pilots  in  command  should  employ  the 
same  style  of  management  in  all 
situations  and  with  all  crewmembers. 

(C42) 

2.72 

2.50 

3.00 

2.92 

2.57 

10-  Pi  lots- in-command  instructions  to 
other  crewmembers  should  be  general  and 
non-specific  so  that  each  individual  can 
practice  self -management  and  can  develop 
individual  skills.  {C43) 

Disagree 

2.83 

2.62 

3.00 

2.92 

2.95 

11,  Training  is  one  of  the  pilot- in¬ 
command's  important  responsibilities. 

(C16) 

WM 

6.12 

6.37 

5.50 

6.62 

6.38 

113.  The  pilot-in-coninand‘s 

responsibilities  include  coordinatiw  of  6.12  6.25  6.46  6.19  J 

+  c##  denotes  corresponding  Army  item  number. 

++  Army  and  Civilian  differ  on  this  item. 

***  p  <  .01,  **  p  <  .05,  *  p  <  .10 

In  general  it  was  found  that  Army  aviators  have  ratings  similar  to  the  NAS  A/UT  commercial 
SS  Furthennore,  the  differences  between  the  high  and  low  groups  of 
SSytote  expected  direction.  Were  the  Army  sample  larger,  it  might  teve  been  po  sible 
m  obtato  additionalSmficant  differences  between  the  means  of  the  h^h  Md  low  ^"P^- 
be  however  that  the  lack  of  significance  could  also  be  due  to  a  lack  of  power  of  te  CI^Q 
to ’discriminate  between  “good’  and  "poor"  Army  pUots. 

samole  size  these  results  must  be  considered  exploratory  and  therefore  subj^t  to  turme 
^ato.  ^r  these  same  masons,  i.e.,  small  sample  and  exploratory  research,  the  typical 

test  of  significance  was  relaxed  fi:om  .05  to  .  10. 

At  this  point,  explanations  as  to  several  responses  are  in  order.  In  respon^  to  Smternent  1,  "My 
decision-making  ability  is  as  good  in  emergencies  as  in  routine  mission  situaUons,  Amy 
aviators  appear^to  differ  from  commercial  aviators.  The  difference  is  thought  to  be  due  to  two 
reasons  FfrsT DRC  was  told  by  the  IP-raters  and  I/Os  at  Ft.  Campbell  that  ^y  aviators  are 
t^t  that  in  emergency  situations  their  abilities  are  heightened.  However,  this  noUon  is  ody 
pailaUy  correct.  Increased  adrenaline  heightens  attention,  strength,  ruction  speed,  but  it 
hso  hi  the  undesirable  effect  of  focusing  attention.  Focusmg,  or  mnnel  vision,  does  not 
enhance  decision  making  ability.  Secondly,  Amy  aviators  are  taught  to  fly  under  adverse, 
dangerous  (wartime)  conditions.  The  objective  of  Amy  aviation  missions  is  aecessanly  twofold 
safety  and  mission  accomplishment  -  a  dichotomy  requimg  careful  balance  and  a 

to  t^e^culated  risks  not  expected  of  commercial  aviators.  h  toe 

different  constraints,  i.e.,  safety  is  toe  primary  consideration;  of  sewndaiy  importan^  is 
delivery  of  passengers  or  freight.  Consequently,  the  Amy  aviator  is  perhaps  more  like  y 
behevlthat  his  or  her  flying  abilities  are  equal  during  both  emergency  situaUons  and  routme 

missions. 

In  resTXinse  to  Statement  4,  "The  pilot-in-command,  time  and  situation  pemitting,  should  t^e 
aircraft  in  uU  mergency  and  non-standard  situadons,'  the  itoy  stahstt^ 
means  are  in  contrast  to  toe  commercial  aviators.  One  reason  for 

that  toe  pilot  in  command  will  be  given  absolute  authority  m  toe  cockpit.  The  high  quality 
group  is  however,  slightly  in  agreement  with  toe  statement.  Both  toe  groupings  (  perfomers 
and  "qukty")  show  separation  in  toe  desired  direction  in  terns  of  toe  pnnciples  of  aircrew 
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tinn  Were  there  a  larger  sample  of  Army  aviators,  it  might  be  found  high 
onSh  "quality"  aviators  align  more  closely  with  the  "superior"  commercial  aviators, 

notwithstanding. 

ct  tamATit  s  "There  are  no  circumstances  where  the  pilot  should  take  the  aircraft 

In  response  to  Statement  5,  tK*.  niint  in  command  "  Army  aviators  felt  similarly 

controls  without  being  dUected  »  i"  -eans  for  the  high 

to  commercial  aviators.  ’  ,  ,  .  •  ^  "nuality"  Army  aviators  disagree  with  this 

"  s'xs.  "  -  ■“  •• 

exi^tS  anTu  good  expStation  for  the  significant  difference  is  not  available. 

ox  »  11  -Trtaininp  is  one  of  the  pilot-in-command’s  important 

Army  response  “  is  essentM  to  maintaining  a  cooperative 

responsibilities,"  and  Statement  2,  A  retoed  amtude  is  e^n  „„eial  aviators. 

and  harmonious  cockpit,"  are  m  a  high  performer 

items  between  high  and  low  "performers. 
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3.1  General 


Section  3.0 

Properties  of  the  ACE  Checklist 


Scales  and  scale  scores  were  created  for  the  ACE  CheckHst.  The  data  collected  at  Ft.  Campbell 
were  based  on  the  first  use  of  the  ACE  Checklist,  the  initial  version  of  which  is  at  Appendix 
A  2  Frequency  distributions  were  developed  for  each  ACE  Checklist  item;  and  item  analyses 
were  peSirmed  for  each  of  the  six  derived  ACE  Checklist  scales.  Frequency  distributions  and 
scale  construction  are  further  discussed  below. 

3.2  Frequency  Distribution 

Appendix  C  contains  frequency  distributions  for  the  ACE  Checklist  items.  [Note.  ACE 
Checklist  items  1-19  are  referred  to  sequentially  as  A1  to  A19.] 

Due  to  pre-testbed  training  to  establish  interrater  reliability,  fte  participating  IPs  were  clearly 
able  to  differentiate  aircrew  performance  in  terms  of  the  behavioral  anchors  associat^  with  each 
ACE  Checklist  item.  The  effectiveness  of  such  training  was  demonstrated  quantitatively  by  the 
raters  availing  themselves  of  the  entire  seven-point  scale.  As  expected,  ratings  were  generally 
in  the  lower  half  of  the  rating  scale  since  the  Fort  Campbell  aircrews  were  not  given  aircrew 
coordination  training  prior  to  the  simulator  mission. 

QuaUtatively,  during  testbed  debriefmgs,  the  IPs  unanimously  attested  to  the  ease  of  use  of  the 
ACE  Checklist  and  the  content  validity  of  the  instrument.  They  suggested  some  fine-tunmg  of 
the  training,  e.g.,  emphasize  that  ACE  items  18  (Management  of  abnormal  or  emergency 
situation)  and  19  (Conflict  resolution)  are  optional.  One  instance  which  attests  to  the 
effectiveness  of  the  training  in  the  use  of  the  ACE  was  exemplified  by  an  IP-rater  who  rapidly 
recognized  a  return  to  previously  used  "norm-referenced”  ratings  versus  the  "criterion- 
referenced"  ratings  required  for  the  testbed.  He  was  able  to  correct  his  ratings  to  reflect  the 
method  taught  during  the  pre-testbed  training. 

Examples  of  other  IP-rater  comments  regarding  the  ACE  Checklist  were  "Anchors  were  helpful, 
well-written,  worked  fine,  very  easy  to  use."  Another  IP-rater  stated  that  The  wording  of  the 
anchors  is  realistic  -  very  good... They  are  good  descriptions.  Nice  spread.  Enough  room  for 
judgements."  The  third  IP-rater  stated  that  "The  seven-point  scale  worked  fine  ...  the  anchors 
were  very  helpful,  absolutely." 

3.3  Scales  and  Scale  Construction 

The  16  aircrew  coordination-related  items  and  three  overall  mission  performance  and  workload 
items  were  used  to  derive  the  six  ACE  Checklist  scales.  For  purposes  of  the  testbed  data 
analysis.  Item  *11  (Overall  Workload)  was  excluded  because  it  was  considered  external  to  crew 
conttol  ’and,  in  the  case  of  the  Fort  Campbell  testbed,  all  crews  were  required  to  fly  a 
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standardized  mission  scenario.  The  derivation  of  the  ACE  subscale  definitions  is  explained  in 
the  Dp.velnnment  nf  Measures  technical  report;  they  are  part  of  the  Resource  Integration  for 
Crewed  Systems  (RICS)  Model  developed  by  DRC  for  ARIARDA  during  the  initial  stages  of 
the  current  project.  The  scale  definitions  are  summarized  in  Table  3.3-1. 


Table  3.3-1 

ACE  Checlclist  Scale  Definitions 


Scale  Name 


Scale  Definition 


1)  Total  ACE 


All  16  aircrew  coordination-related 
items  on  the  Checklist. 


2)  Establish/maintain 
team  relationships 


3)  Cross  monitoring  of 
crew  performance 

4)  Mission  Information 
Exchange 


5)  Establish/maintain 
reasonable  workload 
levels 

6)  Global  performance 


Establish  and  maintain  interpersonal 
relationships  to  create  and  maintain 
a  harmonious  team  atmosphere  and  to 
execute  mission  objectives. 

Check  each  other's  actions  for  possible 
errors . 

Establish  and  maintain  the  same  mission 
plan  and  a  common  frame  of  reference 
within  each  crewmember's  mind  in  as 
much  detail  as  possible.  Expose  the 
decision-maker  to  the  full  range  of 
action  options  available  at  each 
important  decision  point. 

Allocate  workload  in  a  reasonable 
manner  across  crewmembers. 


Global  judgements  of  crew  technical  and 
resource  management  effectiveness. 


During  the  development  of  the  ACE  Checklist,  each  item  was  logically  placed  into  one  of  five 
behavioral  domains.  In  the  initial  construction  of  the  ACE  subscales,  ACE  item  ^1  was  placed 
into  more  than  one  domain  (see  Table  5.2-1  in  the  Development  of  Measures  technical  report). 
As  discussed  previously  in  relation  to  the  Army  CMAQ,  for  purposes  of  the  data  analysis  it  was 
necessary  to  make  the  subscales  distinct  to  ensure  that  the  correlations  and  regressions  utilizing 
the  subscales  were  inteipretable.  Consequently,  Item  '1  was  placed  under  the  "Establish/Maintain 
Team  Relationships"  behavioral  domain.  Placement  of  the  items  uniquely  within  each  behavioral 
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domain  is  presented  in  Table  3.3-2.  A  sixth  domain,  which  is  a  global  rating  comprising  the 
overall  technical  proficiency  and  crew  effectiveness  dimensions,  is  also  included  in  Table  3.3-2. 


Table  3.3-2 

Placement  of  ACE  Checlclist  Items 
Within  Behavioral  Domains 


■Qr.ale  fPomainl  Name  ACE  Item  Numbers 

1)  Total  1/  2,  3,  4,  5,  6,  1,  8,  9,  10,  11,  12 

13,  14,  18,  19 


2)  Establish/maintain  1,  9,  18,  19 

team  relationships 


3)  Cross  monitoring  of 
crew  performance 

4)  Mission  Information 
Exchange 

5)  Establish/maintain 
reasonable  workload 
levels 


7,  8 

2,  3,  4,  5,  6,  10 
11,  12,  13,  14 


6)  Global  Performance  15,  16 


All  ACE  items  were  answered  using  a  7-point  scale  ranging  from  Very  Poor  (value  =  1)  to 
Superior  (value  =  7).  Results  of  the  reliability  analyses  (Cronbach’s  Alpha)  performed  on  each 
of  the  six  scales  are  presented  in  Table  3.3-3,  together  with  the  average  item  and  scale  scores. 
A  correlation  matrix  computed  for  the  six  ACE  Checklist  scales  is  presented  in  Table  3.3-4. 
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Table  3.3-3 

Cronbach's  Alpba  and  Scale  Scores 
for  the  ACE  Checklist  Scales 


Rt-.ale  Name 

1)  Total 

2)  Establish/maintain 
team  relationships 

3)  Cross  monitoring  of 
crew  performance  ^ 

4)  Mission  Information 
Exchange 

5)  Establish/maintain 
reasonable  workload 
levels 

6)  Global  performance 


(n=20) 

Nximber  of  Cronbach '  s 
Items  Alpha 

16  .93 

4  . 66 

2  .69 

6  .69 

4  .83 

2  .90 


Avg.  Item 

Scale 

Score 

S.D. 

3.30 

.73 

3.59 

.78 

3.28 

.98 

3.15 

.76 

3.26 

.86 

3.28 

.98 

Table  3.3-4 

Correlations  Between  ACE  Scales 


Total 

Team 

Reis 

Cross 

Monitor 

Info  Workload 
Exchanoe  Mot. 

Total 

— 

Team  Reis. 

.90 

— 

Cross  Monitor 

.86 

.74 

— 

Info  Exchange 

.91 

.73 

.76 

— 

Workload  Mgt. 

.87 

.75 

.66 

.67  - 

Overall  Perf. 

.89 

.81 

.75 

.76  .85 

Overall 
Perf . 


Subscale  reliabiUties  (internal  consistency  as  measured  by  Cronbach ’s  Alpha)  for  the  ACE  ^e 
high;  reUability  for  the  entire  instrument  is  exceptionally  high.  Furthermore  the  - 
correlations  among  the  subscales  were  also  very  high.  In  summary,  crews  rat^  as  high  or  low 
tended  to  be  consistently  rated  as  such  across  the  items  and  across  the  subscales. 
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3.4  APF.  rhecklist  Factor  Analysis 

A  factor  analysis  was  performed  on  the  ACE  checklist  data  to  determine  any  underlying 
components  of  the  measure.  An  unconstrained  varimax  rotation  factor  analysis  yielded  a  four 
factor  model.  Upon  inspection,  the  fourth  factor  provided  results  that  could  not  be  meanmgfoUy 
interpreted.  It  was  determined  that  a  simpler,  three-factor  model  might  be  more  understandable. 
Thus  the  factor  analysis  was  constrained  to  three  factors  and  the  resultant  data  were 
inteniretable  and  reasonably  labeled.  The  rotated  (varimax)  factor  matrix  converged  m  7 
iterations  with  72.5%  of  the  variance  explained.  Table  3.4-1  shows  the  factor  loadmgs  for  the 
three  factor  model.  Note  that  A1  through  A19  correspond  to  the  ACE  Checklist  item  numbers 

1-19. 

As  depicted  by  Table  3.4-1,  the  three  factors  were  similar  to  the  previously  defined  ACE 
subscales.  Factor  1  was  determined  to  be  an  indicator  of  communication  and  group  climate, 
Factor  2  was  presumed  to  be  an  indicator  of  workload  disttibution  and  performance 
management;  while  Factor  3  appeared  to  be  best  explained  as  indicating  cross  monitoring  by 
crewmembers. 


Table  3.4-1 

Factor  Analysis  Results 
for  the  ACE  Checlclist 

(n=20) 


FACTOR  1 

FACTOR  2 

FACTOR 

3 

( Communication 

(Workload  & 

CE  Item  #+ 

£c  Gro  Climatel 

Perf.  Mot.l 

f Cross  Monitor) 

Al 

.11125 

. 40173 

.51078 

★ 

A2 

.52846  * 

.42282 

.57056 

A3 

.69440  * 

-.04313 

.63908 

A4 

.70121  * 

.02953 

.34031 

A5 

.71185  * 

.46635 

.07074 

A6 

.67423  * 

.14969 

. 13009 

A7 

.48828  * 

.47881 

.20985 

A8 

.56907 

.15701 

.65959 

* 

A9 

.80547  * 

.18787 

.06974 

AlO 

.49472 

.58456  * 

.36848 

All 

.08832 

.86557  * 

.25846 

A12 

.02888 

.75195  * 

.32069 

A13 

.06652 

.  27261 

.90594 

★ 

A14 

.30229 

.25411 

.84659 

* 

A15 

.33973 

.71600  * 

.38310 

A16 

.44989 

.61096  * 

.56115 

A18 

.27158 

.86927  * 

-.04764 

A19 

.61267  * 

.46395 

.28839 

*  "Best  Loading"  determined  by: 

1)  factor  loading  and 

2)  if  an  item  loaded  closely  on  two  factors,  it  was  logically 
placed. 

+  ACE  Items  #  A1  to  A19  are  ACE  item  numbers  1  to  19. 


Further  use  of  the  ACE  "factor  analytic"  scales  was  rejected  for  several  reasons: 

1)  The  factors  derived  from  the  factor  analysis  were  similar  to  the  logical  scales  constructed  and 
discussed  in  the  Development  of  Measures  technical  report. 

2)  Results  of  any  factor  analysis  from  a  sample  as  small  as  the  testbed  sample  must  be  viewed 
with  caution.  When  the  Army  CMAQ  "factor"  scales  were  created,  the  NASA/UT  (Gregorich, 
et  al.  1990)  study  was  available  in  the  literature.  This  study  provided  sufficient  corroborating 
evidence  from  a  much  larger  sample  that  the  DRC/ARIARDA  CMAQ  "factor"  scales  were 
robust.  Furthermore,  Gregorich  (personal  communication,  1991)  stated  that  when  NASA/UT 
performed  a  factor  analysis  on  the  Line/LOFT  Worksheet  (an  instrument  similar  to  the  ACE 
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?eS^  gtl  fte  smaU  aaLple  size  and  lack  of  any  cortoboradng  evidence. 

31  The  underlying  factor  structure  of  the  Army’s  ACE  data  may,  in  fact,  be 

:r:^cmi;^e to  ^m^rof” 

Strcr^^t“to"  dirtriSrshow  up  as 

important  dimensions. 

41  There  was  no  improvement  in  depicting  scale  relationships  through  the  use  of  toe  ACE  ^tor 

"factS^rdetermine  if  there  were  improvements  in  the  corrdation 

relationships  between  the 

ACE  and  external  variables. 

5)  There  was  only  a  ^atom  toe  «  — ^ 

rJS^Smty^fficirrntwl^erA^crQ  -logi^-.sc^es  did 

to  nave  mg  .f,Mnr  analvtic”  scales  revealed  Alpha  coefficients  for  Factors  1,  2,  Md 

*  toese  coefficients  are  better  than  those  for  the  ACE 

"togi;  biisS^^’  ^es,  toe  -logic4ed;  scale  reliability  coefficients  had  revealed  themselves  to 
be  well  within  the  range  of  acceptability. 

Tn  cnmmnrv  hecausc  a  larger  sample  was  not  available,  corroborating  evidence  of  the  stability 
S  Ae  ace’  ’^factor  analytic"  scales  could  not  be  shown,  and  because  there  was  no  m^k^ 
improvement  in  correlations  with  external  variables,  it  was  detemined 

existed  to  incorporate  the  ACE  "factor  analytic"  scales  into  subseqeunt  analyses.  Consequently, 
the  "logic-based"  scales  have  been  used  in  this  analysis. 
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4.1  Gener^ 

C  loc  «!ale  scores  were  created  for  the  revised  Aircrew  Training  Manual  (ATM)  Tasks. 
The  data  collected  at  Fort  Campbell  were  based  on  the  first  use  of  the  revised  a™  Tas  ^ 
S^Modifi^  Grade  Slips.  The  Modified  Grade  SUps  developed  to  ^pture  performance  dato 
mmnrisine  both  technical  and  aircrew  coordination  components  can  be  seen  m  Appenda  .  . 
PrSnoy  developed  for  each  Modified  Grade  SUp  tem;  and  .tem  « 

wS  pe^rmed  for  each  of  the  derived  a™  scales.  Frequency  distnbuhons  and  scale 

construction  are  further  discussed  below. 


Annendix  D  Contains  frequency  distributions  for  the  Revised  ATM  Task  items.  pSTote:  The 
Revis^  ATM  Tasks  are^fen^  to  as  TW#  where  represents  the  f™.'’ 
nnl^S  Of  the  task  in  TC  1-212,  the  ATM  for  the  UH-60A.]  The  frequency  distribuhon  tablB 
for  the  ATM  Tasks  are  arranged  in  task  numeric  order,  low  to  high;  and,  smce  only  those  tas 
toclIlL  ™  e  fmtodized  ienario  developed  for  the  testbed  were  rated,  gaps  m  the  numerical 
order  of  the  ATM  frequency  tables  result. 

At  thk  noint  it  is  necessary  to  explain  the  methods  used  for  the  selection  of  ATM  Tas^  to  be 
included  in  the  testbed.  For  those  ATM  tasks  hav^ 

rd““i?-c— 

diirinB  the  testbed  was  1002  (IFR  Flight  Planning).  This  resulted  in  two  ATM  Taste  being  rated 

itiirinv  the  testbed  Other  ATM  taste  appearing  in  the  frequency  tables  which  have 
Sn^on  ^^deradoL  were  sele«ed”n  accident  data  obtained  from  the  Army  S^e^ 
Management  Information  System  (ASMIS).  By  this  method,  if 

Mnc  nerformed  immediately  prior  to  the  emergency  situauon  and  resulted  in  either  five  Ctes 
A  ai^tots  or  a  total  of  ten  Class  A,  B,  and  C  accidents,  then  it  was  select^  for 

toncritici  aircrew  coordinatton  requirements  identified  during  ai^iden^^^^^^^^ 

absent  or  lacking.  This  process  resulted  in  the  selecuon  of  an  addiuonal  12  A™  Tas^  tor 
rewrite  11  of  which  were  rated  during  the  testbed.  Thus  there  was  a  of  13  aircre 
A-  ’orinn-TPiatprl  ATM  Tasks  used  in  the  testbed.  The  remaining  16  ATM  Tasks  of  the  29 
k  mtfrequency  tables  were  included  to  obtain  non-aircrew  coordination-related  performance 
data  and  to  enhance  reliability  of  the  performance  measures. 

Unlike  the  other  frequency  tables  in  Appendix  D,  Table  #1  (BIGRADE)  relates  to  the  global 
erade  assigned  by  the  IP  to  the  crew  after  considering  both  the  technical  and  the  mere 
?SSnC^mL  of  the  rated  taste.  For  the  testbed,  a 

Proficiency  and  Readiness  Test  (APART)  rating  procedure  was  made,  ^orm^ly,  j^^sing 
standard  field  rating  system  of  satisfactory  or  unsatisfactory,  failure  o  any  one  s 


28 


apart  flight  evaluation  would  result  in  an  unsatisfactory  grade  ta  th^ght.  to  that 

was  ha^  »  ui^^facto^ 

Xtfto“gita"rade  » the  flight  which  best  reflected  overaU  performance.  Table  #1 
reflects  this  grading  technique. 

*  tv,  APP  rhecklist  observation  on  the  effectiveness  of  pre-testbed  evaluator  training, 

tog“rSeTnit  avLtors  in  aaa<^^ 

rs^ev^rcr  .  .e  righl  Pl^  and 

appropriate. " 


les  and 


action 


Twenty-nine  ATM  Task  items  were  used  to  derive  two  ATM  s^es 

ATM  Tasks,  the  second  includes  only  the  revised,  aircrew  coordma  on  „  •  ^ata" 

^™dent  tough  inspection  of  the  frequency 

problem;  i.e.,  for  many  of  the  which  SPSS-PC 

analysis  for  WSr^^tmpuS  “  Sle^»«  itams  having  a  complex 
operates,  Cro^h  ^  Alpha ^  ^  T^sk,  then  that  case  is 

set  of  responses,  i.e.,  um-  i  Th#»rpfnre  sckle  reliability  was  calculated  using 

eUminated  SS  of^Stm^having  a  complete  se, 

®  *a"  formula  was  appUed  to  determine  the 

reliability  for  the  lengthened  test. 

is  expressed  as; 


Reliability  of  lengthened  test  = 


kr  ■_ 

1  -H  [(k-l)r] 


Where,  k  is  the  changed  test  length 

r  is  the  reliability  of  the  original  test 

ReliabiUty  of  the  ATM  scales  was  very  good.  Of  the  29  ATM  ‘I'o 

standardized  testbed  mission  scenario,  19  had  complete  data  sets.  Cronbac  s  P  “  .y  j 
items  was  computed  as  .85.  Applying  the  Spearman-Brown  prophecy  formula,  the  rehabiUty 

the  29  items  is  estimated  to  be  .90. 
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1C  •  ^  atm  Tasks  13  received  ratings  during  the  testbed  with  10  items  having 
Of  the  15  revi  .  ^  ^  the  10  items  was  computed  as  .79.  Applying  the 

"o-ll  me  reuability  for  me  13  items  m  me  msibed  ts 

eSted  to  be  .83.  For  the  15-item  scale,  reUability  is  estimated  to  be  .85. 

1  tViP  A  "R  r  or  U  scoring  method  to  rate  each  of  the  ATM  Tasks.  These 
IP-raters  employed  the  A,  B,  C.  or  u  sco^g  mcu  ^  C  =  2,  and  U  =  1.  Scale 

scores  were  converted  to  a  four  pomt  seal  ATM  Tasks  were 

^es  for  me  29  ATM  Tasks  and 

ooordinadon-relamd  ATM  Tasks 

was  2.48  (S.D.  =  .47). 

aircrew  coordination-related  seal  ^  ^  the  single 

:im^“"m"('^erall  Grade  -  -BIGRADE-  in  me  Appendix  D  ftequenoy  table); 
and  ATM  Task  1071. 


Table  4.3-1 

Modified  Gradeslip/ATM 
Bubscale  Correlations 

(n=20) 


All  ATM  Tasks 

All  ATM 
Tasks 

AC  Tasks 

AC  Tasks 

AC  Tasks  minus 

.93 

Task  1071 

.92 

.  99 

Overall  Grade 

.85 

.  86 

Task  1071 

.65 

•  66 

AC  Tasks  Minus  Overall  Task 
T^sk  1071  Gra^  IQIX 


performance  on  me  ATM  Tasks.  However,  as  mentioned  ^ove,  a  ^  deal  of 

me  ATM  Tasks  in  meir  revised  format  was  easy  and  mat  me  revisions  had  a  great 

content  validity. 
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,TO  Task  1071.  "P=*” 

““Cft  Jwi 

performance  (Table  4.4-1). 


ci-ar>dar<L 


Table  4.4-1 
TasX  1071  Standards 

Text  ^ 

actively  participate  in  the 

»i-irn,"ira^"=?.pusl>ea  prior 

A  detailed  aircrew  brietmg 

to  takeoff.  acknowledges  his  role. 

Each  and  tasL  for  the  entire 

responsibilities, 

mission.  ,  4.  •  i  <?  established  and 

ri;?aln”nrstandara  phraseology  and  visual 

Si^lfes  are  encouraged  -d  iudi=i°-^V  resolved 

rxi^eslSriinnio-^-  ie  Shared  between 

crewmembers.  participate  in  the  problem  solving 

^2.1  crewmembers  parti  p 

process.  .  demonstrated  at  all  times 

Situational  awareness  i  to  mission 

by  each  e'^ewm^er  wit  equipment  status, 

fn^ifonruiarrorditi^ons.  and  personnel 

°,fl^"ci^ei^i^ers  coordinate 
ensure  that  critical  tasx  uxiu 

achieved.  ruinate  in  the  critique 

^^icfsS™r^?ter?nr=ri?iciam  in  a  constructive, 

J^ISSImrelsTorrsmoothly  as  a  team  committed  to 

safe,  mission-oriented  flying. 


- -  TTIttTt^  revised  for  this  Project  included  both 

Sw  o^ta"des  (arhculated  in  me  Talk  Smdards  and  Descnphon) 
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.  j  v;t,o  -Pmnlov  aircrew  coordination  techniques  in  accordance  with  Task 

debrief  that  referencing  Task  1071  Standards 

in  other  ATM  Tasks  "worked  fine." 

au  «f  thP  ATM  Task  1071  Standards,  when  an  aircrew  was  given  a  B,  C, 
With  respect  to  the  use  of  the  A  aririitional  items  of  information.  The  first  item 

or  U  rating,  IPs  were  reqmred  to  „bted  to  flight  or  aircrew 

performed  during  the  simulator  scenario. 


ATM 

Task#+ 


1001 

1007 

1015 

1017 

1028 

1031 

1068 

1071 

1098 

2009 

2016 

2081 

2084 


Total 


Table  4.4-2 

,  TP-Rater  Use  of  Task  1071  Standards 

“  r  cr.V  Coorain.tion-Ml.t.a  R.vis.a  ATM  Tasks 


SI 

3 


1 

5 


SI 

6 


11 


S3 

4 
3 

1 

1 

5 
2 


19 


1071,  Standard :.4-+ 


Si  S5 


Si 

3 


SI  Si  Si 


3 

4 

10 

2 

3 

4 
7 


34 


2 

1 


6 

5 

13 

1 

3 

4 

5 
2 

45 


44 


8 


SIO  Sii 
2 


3 

1 

3 

1 

2 

3 

2 

3 


23 


+  ATM  Task  #  refers  the  ATO  4"  ^SfeUven 'standards 

++  Task  1071  Standard  SI  to  Sll  refer  ro 

listed  in  Table  4.4-1. 


were  rated  on  Task  1028  (Perform  ^MC  Approach).  No  T^^71StMtodsj 
on  Tasks  1015  and  1028  because  deficiencies  in  performance  on  those  tw 
rated  as  flight  skill-related  and  not  aircrew  coordmaUon-related. 
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As  the  Table  4.4-2  Other  highly  used 

Standard  8.  One  IP-iater  commented  that  Standard  5  was  y 

Standards  were  4,  3,  2,  1  and  9. 

J  U  C4S  !.ird*rews  did  not  avail  themselves  of  the  opportunity  to 

standard  10  was  not  “*f.f^^““^'<lthere  was  Utfle  time  during  the  emulator  session 

accomplish  their  own  post-flight  d^nefing  ^  ^  the  critique  funcUon 

to  cridque.  Also,  it  is  noted  that  ther^  !JXIo7l“5l^A  has  included  statements 
(during  or  post-mission).  .  ^sed  ATM  Task  1098  (After  Landing  Tasks); 

covering  the  post-flight  critique  scenario  and  hence,  was  not  graded.  Also,  in 

however,  this  Task  was  not  ^  h  would  rate  crew-initiated  critique  during 

«  S^e^Sbrilltrdid  not  a^  »  emlKuly  tlds  very 

important  function. 

During  actual,  non-simulated  operations,  ““'^^.'’^^g)jField  refueling  might 

other  appropriate  times  of  an  operation  (  “  ^  ,  2q  seconds,  thus  negating  this 

ake  20  minutes  whereas  m  the  snnulator  re^ng  ^  Task 

opportunity.  Finally,  critique  d^  no  ^  c^andard  is  important,  stated  that  under 

importance  of  Standard  10. 

sandard  5  was  used  on  only  3  —  p^h^y  b-u. 

within  the  crews.  Furthemore,  th  atmosphere  of  mutual  respect."  During  the  post- 

■DiSnces  of  opinion"  instead  of  the  word  "Conflica  . 

Standard  7  was  used  on  only  6  occasions.  It  is  XnS^""a^'yKS 

technical  «P°«-rr“?'::,SSvTg  r^^^^  aircraft 

solving  and  there  is  little  of  that  tj^  o  P  ip-raters  did  use  Standard  7  in  an  amount 

in  a  tightly  coupled  envnonmen^^^  mvfston  "aated  that  Standard  7  is  "good, 

sufficient  to  warrant  its  remaining  in  the  A  i  M  revision,  u 

easily  interpretable." 

in  summary,  the  Revised  ATM  Tasks  »d  ^sc^  ^t/XTdi^^m^  A^^s" 

r  s  rr  Ss^=rwS^.^- ver^b  r^iuty. 


33 


^  .  «f  nur^uinp  research  within  a  simulator  environment  is  the  ability  to 

A  primary  advantage  of  pursu  g  testbed  the  same  set  of  mission  parameters 

standardize  a  mission  across  aew  .  testbed  aircrews.  As  has  been  previously  shown, 

confronted  each  of  the  twenty  .  P  varied  from  crew  to  crew  as  measured  by  the  IP- 

performance  wilhin  the  ACE  Checklist.  In  addition  to  the  IP-ratings 

outcomes  can  be  made. 

.  Only  an  overview  of  the  perform^ce  ^sEsiran^^ed^^^^^  videotapes  of  the 

is  due  to  personnel  from  Anacapa  Sciences  ^  assumi  tha^  Anacapa  has  provided 

missions  to  capture  the  perform^ce  ^anab  *  properties  of  the  performance 

ARIARDA  with  a  detailed  re^rt  d^ussing  variablL  developed  by  Anacapa  used 

Lte  *  Tabt  5.0^2  "brief  deLtions  of  each  of  the  performance  variables. 
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Table  5.0-1 

Utility  Helicopter-Related  Performance 
Variables  Used  in  the  Data  Analyses 


Time  Number 
Cross  of  Fit. 
Crew#  PLOT 


1 

2 

3 

4 

5 

6 

7 

8 

9 

10 
11 
12 

13 

14 

15 

16 

17 

18 

19 

20 


32.0 

27.4 
32.8 

42.0 

33.4 

24.1 

24.3 

27.7 

25.4 

29.8 

26.8 

30.1 
33.0 
30.0 
28.7 

23.2 
38.9 
37.4 
28.6 


2 

1 

2 

2 

3 

1 

3 

0 

1 

2 

0 

1 

2 

2 

1 

0 

3 

2 

1 


%  Time 
off 


7.1 

6.5 

17.8 

37.3 

43.8 

15.8 

28.7 

0.0 

10.4 

34.4 
0.0 
39.2 

21.7 

75.8 
23.0 

0.0 

55.6 

43.8 

32.9 


#of 

Threat 


0 

1 

0 

0 

0 

1 

0 

0 

0 

0 

0 

1 

0 

0 

0 

1 

0 

0 

1 


7 

2 

2 

9 

7 

1 

2 

7 

1 

5 

1 

5 
1 

3 

4 

3 

4 

6 
10 


Total 

Duration 


81 

15 

19 

129 

120 

20 
11 
73 
9 

67 

5 

63 

5 

69 

58 

36 

47 

79 

126 


Mean 

Duration 
of  Longest 

ILS 

Steps 

ILS  % 

Duration 

Encounter 

Correct 

Correct 

11.6 

25 

(7/11) 

64 

7.5 

10 

( 3/12) 

25 

9.5 

15 

- 

* 

- 

(12/12) 

100 

14.3 

25 

- 

• 

17.1 

56 

(9/11) 

82 

20.0 

20 

(12/12) 

100 

5.5 

8 

(9/11) 

82 

10.4 

16 

(12/12) 

100 

9.0 

9 

(7/11) 

64 

13.4 

32 

(10/12) 

83 

5.0 

5 

(11/12) 

92 

12.6 

20 

( 8/12) 

75 

5.0 

7 

(10/11) 

91 

23.0 

46 

(8/11) 

73 

14.5 

23 

(11/12) 

92 

12.0 

14 

( 9/12) 

75 

11.8 

24 

(11/12) 

92 

13.2 

23 

(11/12) 

92 

12.6 

23 

(11/12) 

92 

Nnte*  Crew  3  mission  aborted  prior  to  ILS  approach.  ... 

cZ  4  Stag  dau  due  to  a  techuical  problem  with  the  videotape. 
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Table  5.0-2 
Brief  Definitions  of 
the  Performance  Variables 


Variable 
Crew  #: 

Time  Cross 
FLOT 

Number  of 
Fit,  Dev*B: 

%  Time  off 
Course: 


Within: 


#  of  Threat 
Encounters : 

Total  Duration 
Encounters 

Mean  Duration 


Duration  of 
Longest  Encounter 

ILS  Steps 
Correct 


ILS  % 
Correct 


ThfSeiStial  identification^nu^er_of^the^teBtbed_aircrew.^^ 

phLrorthe^mi^ion!’”  Ideally',  crews  should  have  taken  24  minutes. 

w  s  an  aircrew  deviated  from  the  planned  flight  path 

The  n^er  ph.U  ofthe  .i.elon  by  preeter  than  500  meters  as 

f SsSlfSt  rnivUeg^n  or  crew  coordination  error. 

^  j  4-vio  ractical  phase  of  the  mission  that  th^ 

Sew'’“roff°coSi^  tri  leeult  of  a  navigation  or  crew  coordination 

error. 

Whether  or  not  the  ■?“LnriSy°rid“fo“" 

allotted  24  minutes.  A  "1"  means  yes,  a  u  »» 

The  number  of  times,  during  the  tactical  phase  of  the  mission,  that  tKe. 

helicopter  was  acquired  by  enemy  radar. 

rhat  the  helicopter  was  acquired  by  enemy 
The  total  time  rn  seconds  that  tne 

radar, 

^hat  thc  helicoptsr  was  acquired  by  enem^ 
«LriS!udi"  a\"l  eve'nte.  calculated  by  dividing  the  Total  Dnratree 
^counters  by  the  #  of  Threat  Encounters. 

The  duration  of  the  single,  longest  encounter  with  enemy  radar. 

There  were  a  possible  twelve  ^^^^^Lfs^th^nul^Ser^of  Itep. 

instrument  approach.  The  thi  Ser  oi  steps  (of  12)  for 

rrafe^i^r  To^cr^wf  hL^f^eSL  than  twelve  steps  due  to 
"unknown"  performance  on  certain  steps. 

The  percentage  of  correct  step,  during  the  instrument  approach. 
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Section  6.Q 

p^lationship^  Amonp  the 


6  1  Introduction 

The  de.  coueced  at  Fort 

exploration.  For  e*  m  dl^XtS  for  fois  type  of  inquiry  by 

“  sln^lSetir  ^y  ™ 

K"aCT  ctSS  Task^onnance,  and  the 

foe  TSfns% 

provided  in  Paragraph  10.10. 

1.  What  is  foe  relationship  between  foe  two  measures  of  crew  behavior  (ACE  Checklist  and 
ATM  Tasks)? 

What  is  foe  relationship  between  crew  coordination  behaviors  (ACE  ChecMist)  and 
Mission  Performance? 

What  is  the  ’relationship  between  crew  behaviors  (ATM  Tasks)  and  Mission  Performance? 

What  is  the  relationship  between  foe  combinrf  effect  of  crew  coordination  behaviors 
(ACE  Checklist  +  ATM  Tasks)  and  Mission  Performance. 

Which  organization  of  the  Army  CMAQ,  'logical-  or  'factor,'  is  better? 

r»;"foti""™;n“  ”  »d\e:  — S 

behaviors/Mission  Performance? 

What  is  the  relationship  between  crew  coordination  attitudes  (Army  CMAQ)  and  crew 
coordination  behaviors  (ACE  Checklist)? 

What  is  foe  relationship  between  crew  coordination  attitudes  (Army  CMAQ)  and  crew 
coordination  behaviors  (ATM  Tasks)? 

What  is  foe  relationship  between  crew  coordination  attitudes  (Army  CMAQ)  and  Mission 
Performance? 

Sl!or*(A™rcSt'r4“'ch“S  +  S  S  and  Mission  Performance? 


2. 

3. 

4. 

5. 

6. 


8. 


9. 


10. 
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ov^iew  and  a  brief  description  of  the  Sections  is  as  foUows: 


Section  6 


Section  7 


Section  8 


Section  9 


A  -h^c  thf*  analvses  conducted  to  determine  the  relationships 
ResuLrf  S  examining  the  relationships  between  dte 
^es  and  those  of  the  behavior  and  performance  measures  are 

described. 

•  -1  trt  thncp  in  Section  6  but  utilizing  the  CMAQ  '  factor 
Investigations  similar  ape  -►  ATM  previously  presented  in 

subscales.  Certain  relationships,  e.g.,  ACE  aim,  p  y  i' 

Section  6  are  omitted. 

"logical"  scores  are  used  to  develop  the  equations. 

Investigations  simifcn  to  those  in  Section  8,  but  utilizing  the  CMAQ  "factor- 
scales  instead  of  the  "logical"  scales. 

Each  Section ^;^th^"';'ur^sd!  MfttX taT Sr^’d^“s‘’I 

are  presented  together  with  more  in  depth  overview  of  the  findings  of  that  section, 

Section  10  presents  a  summary  of  lhe  analyses  presented  in  this  report. 

With  respect  to  tlte  sample  sizes  used  to  demrmine 

should  be  noted  that  they  vaned  ™  ^  cMAQs  were  available  for  use  in 

aviators  participated  m  the  Fort  ®  checklist  AM  Task  (Modified  Gradeslips)  and 

die  analysis;  however, there  ^  "i„g  data  further  Umited  the  available 

simulatorperformanceo^a^m^somem^^^^  ^  ^  ^  ^ 

sample  size  for  an  equation  Th  ■  “  ®  ^ta  this  may  not  be  precise  for  all 

^^nf  "wt  m  nom’is  L 

tosed;"  when  "(n=40)"  is  cited,  the  analysis  is  "mdividual-based. 

f  ♦u-c  rioto  etnaivik  was  exoloratorv  and  the  sample  size  small,  it  was  decided 
Since  °  of  significance  to  .10.  Tests  of  significance  are  noted  in  the 

in  the  Tables.  In  Sections  8  and  9,  the  criteria  for  F-to-enter  and  F-to-remove  into 
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1  A  to  ciinw  meaninEful  stepwise  regression  equations  to  be 
mis  manspuiation  is  presenmd  in  Section  8. 

In  that  a  great  number  of  Mef'de^^^^^  thtm.  Several 

^^“%^"gSrSmbinsmons  of  me  PC  and  PI  CMAQ  scores,  are  mtroduced  m 

Table  8.1-1. 


Tabltt  6.1“1  . 


Source 
Army  CMAQ 
••Logical” 
scales 


Variable 


TEAMCMAQ 


CREWFAL 

GIVEGET 

HLPCMAQ 

CMQALL 


p-r-ipaf  Desf-T-iption 


•.-4--5nrf  at  the  seventeen  CMAQ  items 
2,0  consistiri^  o  a  5*7# 

elevant  to  Values  J.eaw  '  3q  42  44, 

,  9,  15,  19,  22,  25,  26,  27,  2^,  ^  / 

5)  •  .  at  the  eleven  CMAQ  items 

cale  6  11,  12, 

elevant  to  Crew  Fallibility  ^ 

,4,  17,  18,  20,  21,  28.  CMAQ  items 

;cale  consisting  Getting  information 

tiims 

iS^vanrt:^f.ovldi“n*,  a^d  A=nQ  Help  (#.s 


Army  CMAQ 
••Factor” 

Scales 

COMMCOR 


SHARLEAD 

STRESS 


CMAQ34 


,  36,  37,  40,  44).  eleven  CMAQ  items 

•ale  consisting  of  ^he  eleven 

aivant  to  Shared  ' 

32,  35,  38,  29.  .  '  <3mQ  items 

:ti;ar.r"“^coVi°t\on"‘'ft  l^essor  B«e=ts 

^a!e“ins?it?ng  of' 'all  34  CMAQ  items  included 
n  the  ••factor”  scales. 
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Table  6.1-1  (Cont.) 


ACE 

Checklist 

TEAMACE 

XMNITOR 

INFOEXC 

WORKMNG 

ACEALL 

global 


•  of  the  four  ACE  items 

^fllvanr^'to  ®  EstaMish/Maintain  Tea. 
Relationships  (#  s  i  ^  relevant 

IfcrosTMonitoling  of  Crew  Perfomanoe  (#'s 

’siali'consisting  «;%tha*^ge1# ' s 

to  Mission  Information  Exchange 

5,  6,  10).  -  four  ACE  items 

?=!evanr?o“lhe^lstablish/Maintain^Reasonable 

workload  I'?''jl®„‘f/,w4een' ace' items  (#'s  1, 
S«le  cons^lsting^cf^-«-„_  ,  ,,, 

“globli  performance  measure  consisting  of  two 
ar-TT  15  and  16). 


Graded 

ATM 

Tasks 


ATMALL 

ATM_13 

ATM_12 

bigrade 

TASK1071 


=ale  consisting  of  the  29  AM  Tasks  evaluated 
“aie’  ^"cln^slstl^T'^  Of  ^  13  aircrew 

"etfh  Aircrew  by  the 


rade 


assigned  to  each  aircrew  on  ATM  Task 


Simulator 

Performance 

Variables 

NAVTIME 

DEVIATE# 

%OFFCOUR 

WITHIN 

THRT# 

THRTIME 


of 


ites. 


tactical  phase  of  flight, 


in 


course 


during 


,er  of  deviations  from  planned 
T\n  *hac*tical  phase  of  flight. 

^ent  of  time  off  planned  course 

iion^fo^'w'^th?"  Allotted  time  limit  for 

S"^°encounters  during  tactical 

lurati^n'of  threat  encounters  during 
tical  phase  of  flight. 
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Table  6.1~1  (Cont.) 

THRTMAX 

MEANDUR 

ILSRIGHT 


Duration  of  longest  threat  encounter  during 

encounters  during 

^er™? "I  °ioiVcr-«steps«  on  the  ILS 
approach . 


ro  property  mrpre, 

“r  Lerfl  tt.  mWlME. 

fSiXo^m.  mm.  mmm.  thrimax.  ondUEAwm  vmobtes. 

are  presented.  Table  6.1-2,  shown  below,  is  the  organizational  chart  for  this  SecUon. 
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Table  6-1-2 

Organizational  Chart  for  Section  6 


Analysis /Equation 


6.2  Predict  ATM 
performance  using  ACE 
scales 


6.3  Predict  ATM 
performance  using  CMAQ 
"logical"  scales 


6.4  Predict  ACE 
performance  using  CMAQ 
"logical"  scales _ 


6  -  5  Predict  ATM 
performance  using  CMAQ 
"logical"  scales  and 
ACE  scales 


6.6  Predict  mission 
performance  variables 
using  ACE  scales 


6.7  Predict  mission 
performance  variables 
using  CMAQ  "logical" 
scales  _ 


6.8  Predict  performance 
variables  using  CMAQ 
" logical"  scales ,  ACE 
scales,  and  ATMALL 


Interpretation 


Table ( s) 


6.9  Predict  performance 
variables  using  ATMALL, 
CMQALL,  and  ACEALL 


CMAQ  "logical"  scales 
are  not  predictive  of 
ATM  performance.  CMAQ 
appears  to  have  a  small 
(ns)  effect  on  AC- 
related  ATM  tasks.  _ 


CMAQ  "logical"  scales 
are  not  predictive  of 
ACE  performance  _ 


More  variance  explained 
than  either  independent 
variable  alone.  CMAQ 
consistently  drops  from 
the  stepwise  equations. 


ACE  scales  have 
moderate  ability  to 
predict  mission 
performance  variables. 
WORKMNG  strongly 
affects  navigation. 


CMAQ  "logical"  scales 
are  not  predictive  of 
mission  performance 
variables 


More  variance  explained 
than  any  independent 
variable  alone.  Various 
ACE  scales  and  ATMALL 
are  predictive  of 
several  mission 
performance  variables. 
HLPCMAQ  enters  as  a 
significant  predictor 
variable. 


A  combination  of 
ATMALL,  CMQALL,  and 
ACEALL  are  predictive 
of  several  performance 
var iab 1 e  s ;  CMAQALL 
consistently  drops  from 
stepwise  equations.  
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6.2  Prp/iirt  ATM  Performance  TTsing  ACE  Scales 


Four  of  the  six  ACE  subscales  were  forced  into  the  regression  equations  to  predict  performance 
on  five  ATM  measures.  The  ACEALL  and  GLOBAL  subscales  were  not  included  in  the 
eauations  The  ACEALL  variable,  if  included,  would  have  made  the  predictor  variables 
dependent  on  one  another.  The  GLOBAL  variable  was  of  limited  interest  because  1)  the  focus 
of  the  current  analyses  is  on  the  four  dimensions  of  aircrew  coordination  captured  via  me 
Checklist*  2)  the  GLOBAL  subscale  was  considered  redundant  to  the  ATM  variable,  BIGRADE; 
and  3)  oriy  one  of  me  two  items  comprising  me  GLOBAL  variable  was  related  to  aircrew 
coordination.  The  focus  of  mese  equations  was  to  determine  me  predictive  power  of  me  ACE 
subscales  on  ATM  Task  performance.  The  five  forced  regression  equations  are  at  Table  6.2-1. 


Table  6.2-1 

Porced  Regraasion  of  ACE  Scale* 
with  ATM  Measures* 

(n-20) 


Equation 

1.1  ATMALL  =,10  WORKMNG  +  .18  XMNITOR  + 

(-.12)  INFOEXC  +  .23  TEAMACE  +  1.02 


2 . )  ATM_13  = 


.21  WORKMNG  +  .21  XMNITOR  + 

(-.19)  INFOEXC  +  .21  TEAMACE  +  .96 


3.1  ATM  12  =  .25  WORKMNG  +  .21  XMNITOR  + 

“  (-.26)  INFOEXC  +  .18  TEAMACE  +  1.17 

4.1  BIGRADE  =  .48  WORKMNG  +  .32  XMNITOR  + 

(-.14)  INFOEXC  +  .002  TEAMACE  +  .03 

5.)  TASK1071  =  (-.14)  WORKMNG  +  .21  XMNITOR  +  • 
46  INFOEXC  +  .41  TEAMACE  +  (-1.00) 


Multiple  R 


.77 


.81 


.79 


.76 


.76 


*  In  all  equations,  F  is  significant  at  the  p  <  .01  level. 


%  Variance 

60 

66 

62 

58 

58 


Table  6.2-1  demonstrates  mat  me  ACE  scales  are  highly  predictive  of  ATM  performance.  By 
entering  me  ACE  variables  into  me  equations  in  a  stepwise  manner,  it  was  thought  mat  me  more 
predictive  independent  variables  would  be  revealed.  Table  6.2-2  shows  mese  stepwise  regression 
equations. 


o 
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labl*  6.2-2 

S'bepwj.ss  Re9r8Bsi.on  of  ACE  Scalss 
with  ATM  Measures* 

(n=20) 


Equation  Multiple  .R 

1. )  ATMALL  «=  .39  TEAMACE  +  .97  -71 

2. )  ATM  13  -  .44  TEAMACE  +  .90 

3. )  ATM“12  -  .38  WORKMNG  +  1.28  -69 

4. )  BIGRADE  =  .64  WORKMNG  +  . 12  *71 

5. )  TASK1071  -  .88  INFOEXC  +  (-56)  -70 

*  In  all  equations,  F  is  significant  at  the  p  <  .001  level. 


%  Variance 


51 

52 
48 
51 
50 


Only  one  independent  measure  entered  each  of  the  equations.  WORKMNG  is  the  single  most 
influential  attribute  of  graded  performance  on  the  12  aircrew  coordination-related  ATM  Tasks 
and  BIGRADE.  Likewise,  TEAMACE  best  predicts  overall  ATM  Task  performance  (ATMALL) 
and  aircrew  coordination-related  ATM  Tasks  when  Task  1071  is  included  in  the  scale 
(ATM_13).  Furthermore,  INFOEXC  scale  best  predicts  Task  1071  performance. 

Next,  the  ACEALL  measure  (16  ACE  items),  was  entered  singly  into  forced  regression 
equations  of  the  five  ATM  measures.  The  results  are  shown  at  Table  6.2-3.  F  is  significant  at 
the  p  <  .001  level  in  all  equations. 


Table  6.2-3 

Forced  Regression  of  ACEALL 
with  ATM  Measures* 

(n=20) 

Equation 

1. )  ATMALL  =  .42  ACEALL  +  1.01 

2. )  ATM  13  *  .47  ACEALL  +  .91 

3. )  ATM^12  «  .42  ACEALL  +  1.10 

4. )  BIGRADE  =  .73  ACEALL  +  (--.20) 

5. )  TASK1071  *  .95  ACEALL  +  (-.93) 


Multiple  R 

.71 

.73 

.66 

.69 

.73 


%  Variance 

50 

53 

44 

48 

53 


*  In  all  equations,  F  is  significant  at  the  p  <  .001  level. 


Table  6.2-3  demonstrates  that  the  ACEALL  measure  is  a  powerful  predictor  of  ATM  Task 
performance.  In  summary,  the  ACE  subscales  and  ACEALL  are  very  good  predictors  of  ATM 
Task  performance. 
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6.3  Prp/^ir.t  ATM  Pp-rformanc?  TTsing  the  CMAQ  "L<pgical" 

Reeression  equations  were  next  calculated  to  determine  the  predictive  abiUty  of  the  Army 
Cji\Q  with^espect  to  the  ATM  scales.  The  four  CMAQ  "logical" 

for  discussion  oHLle  development)  were  forced  into  regression  equations  of  the  five  Am 

m"si«ce  CMAQ  ratings  are  individual-bas«i  (rated  ATM 

all  40  cases  were  included  in  these  analyses.  The  five  equations  are  at  Table  6.3  . 


Tabl#  6*3*-l 

Forced  Regreesion  of  CMAQ  "Logical"  Scales 
with  ATM  Measures* 

(n»40) 


Equation 


Multiple  R 


%  Variance 


1.) 

ATMALL  * 

(-.004)  HLPCMAQ  +  .05  CREWFAL  + 

.03  TEAMCMAQ  +  (-.09)  GIVEGET  +  2.52 

• 

to 

1 

2.) 

ATM_13  = 

.008  HLPCMAQ  +  .18  CREWFAL  + 

.04  TEAMCMAQ  +  (-.18)  GIVEGET  +  2.37 

.25 

6 

3.) 

ATM_12  = 

:  .04  HLPCMAQ  +  .17  CREWFAL  + 

.04  TEAMCMAQ  +  (-.22)  GIVEGET  +  2.51 

.27 

7 

4.) 

BIGRADE  =  (-.11)  HLPCMAQ  +  * ^ ®  +  o  49 

(-.06)  TEAMCMAQ  +  (-.10)  GIVEGET  +  2.4y 

.18 

3 

5.) 

TASK1071 

=  (-.24)  HLPCMAQ  +  .32  CREWFAL  + 

(-.03)  TEAMCMAQ  +  .21  GIVEGET  +  .99 

.22 

5 

*  In  all  equations,  F  is  jnot  significant  at  the  p  <  .05  level. 

Table  6.3-1  demonstrates  that  the  CMAQ  Togical’  subscales  do  not  have 
power  when  entered  onto  the  ATM  measures.  Upon  closer 

CMAO  may  have  some  small  effect  on  the  aircrew  coordination-related  ATM  Taste 
(s^l^fically!  ATM_13,  ATM_12,  and  TASK1071);  however,  the  effect  is  not  statistic^y 
significant.  Stepwise  regression  (with  the  probability  of  F-to-enter  ^t  at  P 
Army  CMAQ  "logical"  subscales  resulted  in  no  variables  entering  the  equations. 

6.4  Predict  ACH  Performance  Using  CMAO  "Txigical"  Scales 

The  four  Army  CMAQ  "logical"  subscales  were  forced  into  the  recession  equations  to  ^nd  *eir 
predictive  power  with  respect  to  the  six  ACE  subscales.  The  six  equations  developed  are  at 

Table  6.4-1. 
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Table  6.4-1 

Forced  Regression  of  CMAQ  "Logical"  Scales 
with  ACE  Measures* 

(n=40 ) 


Ecmation 

Multiple  _R 

%  Variance 

1.) 

ACEALL  =  (-.33)  HLPCMAQ  +  .13  CREWFAL 

+  .01  TEAMCMAQ  +  .10  GIVEGET  +  3.86 

.21 

4 

2.  ) 

TEAMACE  =  (-.21)  HLPCMAQ  +  .06  CREWFAL 

+  .05  TEAMCMAQ  +  (-.01)  GIVEGET  +  4.32 

.14 

2 

3.) 

XMNITOR  =  (-.32)  HLPCMAQ  +  (-.06)  CREWFAL 

+  .09  TEAMCMAQ  +  .05  GIVEGET  +  4.69 

.17 

3 

4.) 

INFOEXC  »  (-.53)  HLPCMAQ  +  .16  CREWFAL 

+  .006  TEAMCMAQ  +  .18  GIVEGET  +  4.39 

.32 

10 

5.) 

WORKMNG  «  (-.13)  HLPCMAQ  +  :21  CREWFAL 

+  (-.05)  TEAMCMAQ  +  .13  GIVEGET  +2.17 

.19 

4 

6.) 

GLOBAL  =(-.36)  HLPCMAQ  +  .15  CREWFAL 

+  (-.22)  TEAMCMAQ  +  .33  GIVEGET  +  3.89 

.18 

3 

*  In  all  equations,  F  is  not  significant  at  the  p  < 

.05  level. 

The  Army  CMAQ  "logical"  scales  have  little  value  in  predicting  either  ACE  Checl^st  ^ab  e 
6.4-1')  or  ATM  Task  (Table  6.3-1)  scores.  However,  upon  closer  inspection  of  Table  6.4-1,  it 
would  appear  that  the  CMAQ  may  have  a  small  effect  on  INFOEXC;  however,  the  effect  is  not 
statistically  significant.  Stepwise  regression  (with  the  probability  of  F-to-enter  set  at  p  .  ) 

using  the  Army  CMAQ  "logical"  scales  and  the  ACE  subscales  resulted  in  no  vanables  entenng 

the  equations. 


5  5  Predict  ATM  Performance  Using  CMAQ  "Logical"  Scales  and  ACE  Scales 
Table  6.5-1  shows  the  regression  equations  when  the  ACE  and 

collectively  for  predicting  ATM  performance.  As  in  Paragraphs  6.2  and  6.3,  the  ACEALL, 
GLOBAL,  and  CMQALL  measures  were  not  used  in  the  analyses. 
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Table  6.5-1 

Forced  Regression  of  ACE  Scales  and  CMAQ  "Logical"  Scales 

with  ATM  Measures* 

(n=40) 


•Rquation 


1.)  ATMALL 


*.10  WORKMNG  +  .06  HLPCMAQ  + 

.01  TEAMCMAQ  +  .04  CREWFAL  + 

.18  XMNITOR  +  (-.09)  GIVEGET  + 
(-.11)  INFOEXC  +  .23  TEAMACE  +  .97 


2.)  ATM_13 


*.19  WORKMNG  +  .05  HLPCMAQ  + 

.02  TEAMCMAQ  +  .16  CREWFAL  + 

.23  XMNITOR  +  (-.18)  GIVEGET  + 
(-.20)  INFOEXC  +  .20  TEAMACE  +  .86 


3  )  ATM  12  =  .24  WORKMNG  +  .03  HLPCMAQ  + 

*'  ”  .03  TEAMCMAQ  +  .15  CREWFAL  + 

.23  XMNITOR  +  (-.22)  GIVEGET  + 
(-.27)  INFOEXC  +  .17  TEAMACE  +  1.36 


Multiple  R 


.78 


.84 


.82 


4 . )  BIGRADE  = 


,48  WORKMNG  +  (-.03)  HLPCMAQ  + 
(-.07)  TEAMCMAQ  +  .17  CREWFAL  + 
.34  XMNITOR  +  (-.15)  GIVEGET  + 
(-.17)  INFOEXC  +  (-.01)  TEAMACE  + 


.64 


.77 


5.) 


TASK1071  =  (-.30)  WORKMNG  +  .18  HLPCaiAQ  + 
(-.10)  TEAMCMAQ  +  .31  CREWFAL  + 

.28  XMNITOR  +  .14  GIVEGET  + 

.52  INFOEXC  +  .48  TEAMACE  +  (-3.99) 


.80 


*  In  all  equations,  F  is  significant  at  the  p  <  .0001  level 


%  Variance 


61 


71 


68 


60 


65 


Table  6.5-1  shows  that  when  the  ACE  scales  and  CMAQ  "logical"  scales  are  combined  Jey 
have  exceUent  predictive  power  of  ATM  performance.  When  Table  6.5-1  is  compared  to  6.2-1, 
it  is  apparent  that  the  combination  of  the  ACE  and  CMAQ  subscales  accounts  for  slightly  mwe 
variance  than  the  ACE  subscales  do  alone.  Interestingly,  the  aircrew  coordmation-related  Am 
scales-  i  e  ATM  13,  ATM_12,  and  Task  1071,  appear  to  benefit  most  from  adding  the  CMAQ 
scales’ to  the  equations.  Consequently,  it  was  thought  that  the  more  predictive  measures  would 
be  revealed  by  entering  the  ACE  and  CMAQ  variables  into  the  equations  m  a  stepwise  manner. 
Table  6.5-2  shows  these  regression  equations. 
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Table  6*5-2 

Stepwise  Regression  of  ACE  Scales  and  CMAQ  “Logical”  Scales 

with  ATM  Measures* 

(n=40) 


Ecruation 


Multiple  R 


1. )  ATMALL  =  .25  TEAMACE  +  .15  XMNITOR  +  .98 

2. )  ATM  13  *.26  TEAMACE  +  .22  WORKMNG  +  .83 
3!)  ATM”12  ®.31  WORKMNG  +  .26  XMNITOR  +  (-.22) 

~  INFOEXC  +  1.33 

4. )  BIGRADE  =.44  WORKMNG  +  .26  XMNITOR  +  (-.09) 

5. )  TASK1071  =.55  INFOEXC  +  .44  TEAMACE  +  (-1.10) 


*  In  all  equations,  F  is  significant  at  the  p  <  .0001  level. 


%  Variance 

57 

59 

59 

57 

56 


While  it  was  previously  found  that  the  CMAQ  alone  has  Uttle  predictive  value  with  respect  to 
either  the  ACE  Checklist  or  the  ATM  Tasks,  when  included  with  the  ACE  subscales,  the  CMAQ 
does  influence  the  equations.  For  example,  in  Table  6.2-2,  wherein  the  ACE  subscales  are 
entered  stepwise  into  the  equations,  the  only  variable  loading  with  ATM_12,  the  dependent 
variable,  is  WORKMNG.  In  Table  6.5-2,  equation  #3,  while  WORKMNG  still  enters  first  m 
the  equation,  it  is  apparent  that  XMNTTOR  and  INFOEXC  are  also  important  factors.  In  Table 
6.2-2  only  three  of  the  four  ACE  scales  enter  the  equation;  however,  in  Table  6.5-2,  when  the 
attitude  subscales  are  allowed  to  influence  the  equations,  all  four  ACE  subscales  enter  at  least 
two  of  the  equations. 

Next,  the  two  overall  measures,  CMQALL  and  ACEALL,  were  forced  into  the  regression 
equations  to  predict  ATM  Task  performance.  These  equations  can  be  seen  in  Table  6.5-3. 


Table  6.5-3 

Poirced  Regression  of  ACEALL  and  CMQALL 
with  ATM  Measures* 

{n=40 ) 


Ecruation  Multiple._R 

1. )  ATMALL  =  (-.004)  CMQALL  +  .42  ACEALL  -*-1.03  .71 

2. )  ATM  13  *  .06  CMQALL  +  .48  ACEALL  +  . 60  *73 

3. )  ATM  12  =  .03  CMQALL  +  .43  ACEALL  +  .95  •  oo 

4. )  BIGRADE  =  .03  CMQALL  +  .73  ACEALL  +  (-.36)  .69 

5. )  TASK1071  =  .34  CMQALL  +  .95  ACEALL  +  (-2.83)  .74 

*  In  all  equations,  F  is  significant  at  the  p  <  .001  level. 


%  Variance 

50 

53 

44 

48 

55 


Table  6.5-3  shows  that  the  combination  of  the  CMQALL  and  ACEALL  scales  account  for 
approximately  the  same  percent  of  variance  as  the  does  the  ACEALL  measure  alone  (s^  Table 
6.2-3),  thus  indicating  that  the  CMQALL  adds  little  to  the  equations.  This  observation  was 
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confirmed  when  the  stepwise  regressions  were  calculated.  In  those  equations,  CMQALL 
consistently  dropped  from  the  equation,  thereby  producing  the  same  results  as  found  in  Table 
6.2-3. 

6.6  Predict  Mission  Performance  Variables  Using  ACE  Scales 

The  ACE  Checklist  subscales  were  forced  into  regression  equations  with  the  9  simulator 
performance  variables.  The  nine  regression  equations  are  at  Table  6.6-1. 


Table  6.6-1 

Forced  Regression  of  ACE  Checklist  Scales 
with  Performance  Measures-t- 

(n-20) 


Equation  Multiple  R  %  Variance 


1-) 

NAVTIME  = 

(-4.27)  WORKMNG  +  1.97  XNMITOR  + 

.74  INFOEXC  +  (-1.72)  TEAMACE  +  41.55 

.67 

46  ** 

2.) 

DEVIATE# 

=  (-.54)  WORKMNG  +  .40  XMNITOR  + 

(-.50)  INFOEXC  +  (-.06)  TEAMACE  + 

3.76 

.60 

36 

3.) 

%OFFCOUR 

=  (-14.50)  WORKMNG  +  7.09  XMNITOR  + 

(-12.15)  INFOEXC  +5.48  TEAMACE  + 

69.87 

.62 

39 

4.) 

WITHIN  - 

.14  WORKEXC  +  (-.22)  XMNITOR  + 

.20  INFOEXC  +  .22  TEAMACE  +  (-.89) 

.59 

35 

5.) 

THRT#  =  ( 

-2.02)  WORKMNG  +1.6  XMNITOR  +  .27 

INPOEXC  +  .02  TEAMACE  +4.62 

.51 

26 

6.) 

THRTIME  * 

(-32.40)  WORKMNG  +  20.39  XMNITOR  + 

(-.92)  INFOEXC  +  3.39  TEAMACE  +  83.81 

.51 

26 

7.) 

THRTMAX  = 

(-3.91)  WORKMNG  +  .98  XMNITOR  + 

(-5.04)  INFOEXC  +  1.65  TEAMACE  + 

40.06 

.39 

16 

8.) 

MEANDUR  = 

(-.82)  WORKMNG  +  (-.48)  XMNITOR  + 

1.21  INFOEXC  +  (-.74)  TEAMACE  + 

15.09 

.22 

5 

9.) 

ILSRIGHT 

=  (-6.98)  WORKMNG  +  (-10.71)  TEAMACE  + 

14.94  XMNITOR  +2.18  INFOEXC  +  90.10 

.61 

38 

+ 

Levels  of 

significance  are:  ***  p  <  .01  **  p  <  .05, 

and  *  p  <  .10. 

Table  6.6-1  demonstrates  that  the  ACE  scales  have  a  moderate  ability  to  predict  mission 
performance.  Although  the  multiple  R  values  are  high,  it  was  difficult  to  obtain  statistical 
significance  because  the  degrees  of  freedom  were  relatively  high  compared  to  the  sample  size, 
i.e.,  with  Crew  #4  "missing",  df  =  4,  14.  To  gain  further  clarity  with  respect  to  these 
equations,  the  ACE  Checklist  subscales  were  entered  into  the  equations  in  a  stepwise  manner 
to  determine  which  subscales  offered  the  most  predictive  power.  Table  6.6-2  shows  these  results. 
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X&bXc  6 ♦ 6—2 

Stepwi.e  Regression  of  ACE  Checklist  Scales 
with  Perforsiance  Variables 

(n=20) 


Multiple  R 


*  Variance 


Rquation 

1. )  NAVTIME  »  (-3.53)  WORKMNG  +  41.80  28 

2. )  DEVIATE#  =  (--58)  WORKMNG  "J;  .54  29 

3  )  %OFFCOUR  =  (-12.68)  WORKMNG  +  68.40  24 

4.)  WITHIN  =  .28  TEAMACE  +  (-.72) 

5  *.  )  THRT#  =  NE' 

6 .  )  THRTIME  =  NE 

7 .  )  THRTMAX  =  NE 

8 .  )  MEANDUR  =  NE 

9 .  )  ILSRIGHT  *  NE 

*  In  all  equations  developed,  F  is  significant  at  the  p  <  .03 

T^hiP  6  6  2  shows  that  two  ACE  subscales  are  good  predictors  for  certain  perform^ce 
measures  Interestingly  in  the  Table  6.6-2  equations,  lower  multiple  R  values  result  m  statistic^ 

fVtA  wo'RICMNG  sub  scale  predicts  performance  for  the  NAVTIME,  ? 

r  OF^CO™  measures^Note  that  sin^  three  of  the  significant  equations  predict  navigation 
performance  evidence  is  provided  that  effective  Workload  Management  mfluences  navigatio 
success.  It  is  also  noted  that  TEAMACE  best  predicts  WITHIN  m  the  same  manner. 

^  7  Predict  Mission  Performance  Variables  Usinp  CMAQ  "T^giQ^"  $calss 


6  7  Prpdirt  Mission  Pertormanee.  va^aolc^  — 

The  CMAQ  subscales  were  forced  into  the  regression  equations  with  the  performance  measures. 
Table  6.7-1  shows  the  resultant  equations. 


NE  =  No  equation  derived. 
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Tabl«  6.7-1 

Forced  Regression  of  CMAQ  "Logical"  Scales 
with  Perforsiance  Measures* 

(n=40) 

Equation  Multiple  R 


1. )  NAVTIME  »  (-2.26)  HLPCMAQ  +  1.13  TEAMCMAQ  + 

.37  CREWFAL  +  (-1.59)  GIVEGET  + 

44.85 

2. )  DEVIATE#  =  .15  HLPCMAQ  +  (-.18)  TEAMCMAQ  + 

(-.14)  CREWFAL  +  .10  GIVEGET  + 

1.73 

3. )  %OFFCOUR  =  .69  HLPCMAQ  +  (-4.42)  TEAMCMAQ  + 

2.06  CREWFAL  +  (-4.68)  GIVEGET  + 
62.95 

4. )  WITHIN  =  (-.25)  HLPCMAQ  +  (-.29)  TEAMCMAQ  + 

.06  CREWFAL  +  .38  GIVEGET  +  .77 

5. )  THRT#  =  (-.11)  HLPCMAQ  +  .20  TEAMCMAQ  + 

(-1.10)  CREWFAL  +  (-.35)  GIVEGET  + 
11.44 

6. )  THRTIME  =  (-4.73)  HLPCMAQ  +  (-2.77)  TEAMCMAQ  + 

(9.16)  CREWFAL  +  -25  GIVEGET  + 

141.81 

7. )  THRTMAX  =  (-2.32)  HLPCMAQ  +  (-2.66)  TEAMCMAQ  + 

4.43  CREWFAL  +2.81  GIVEGET  +  9.21 

8. )  MEANDUR  =  (-1.40)  HLPCMAQ  +  (-2.36)  TEAMCMAQ  + 

2.13  CREWFAL  +  1.29  GIVEGET  +  14.22 

9. )  ILSRIGHT  =4.25  HLPCMAQ  +4.66  CREWFAL  + 

4.41  TEAMCMAQ  +  (-10.97)  GIVEGET  + 
74.64 


.34 


.14 


.17 


.36 


.27 


.19 


.20 


.24 


.34 


*  In  all  equations,  F  is  not  significant  at  the  p  <  .05  level. 


%  Variance 


12 

2 

3 

13 

7 

4 
4 
6 

12 


Table  6.7-1  further  corroborates  the  finding  that  the  CMAQ,  by  itself,  has  little  predictive  value 
predicting  performance  in  this  context.  Since  the  percent  of  variance  explained  is  not  significant, 
stepwise  regressions  were  not  computed. 

^  Predict  Performance  Variables  Using  CMAQ  "Logical"  Scales.  ACE  Scales,  and 
ATM  ALL 


The  ACE  subscales,  CMAQ  subscales,  and  the  ATMALL  measure  (29  ATM  Task  scale)  were 
forced  into  the  regression  equations  with  the  nine  simulator  performance  variables.  ATMALL 
was  used  in  lieu  of  the  various  ATM  subscales  because  use  of  the  ATM  subscales  would  have 
introduced  interdependency  among  the  predictor  variables  rendering  the  results  uninterpretable. 
Table  6.8-1  shows  the  resulting  regression  equations. 
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Table  6.8-1 

Forced  Regression  of  CMAQ  "Logical**  Scales, 
ACE  Checklist  Scales,  and  ATMALL 
with  Perfonuince  Neasures+ 

(n*40) 


lation 


Multiple  R 


1. )  NAVTIME  =  (-2.87)  ATMALL  +  .87  TEAMCMAQ  + 

(-2.63)  HLPCMAQ  +  (-.68)  INFOEXC  + 

1.90  CREWFAL  +  (-1.28)  GIVEGET  + 

(-3.62)  WORKMNG  +2.82  XMNITOR  + 

(-1.24)  TEAMACE  +54.72  .77 

2. )  DEVIATE#  =  (-1.30)  ATMALL  +  (-.24)  TEAMCMAQ  + 

(-.03)  HLPCMAQ  +  (-.66)  INFOEXC  + 

.20  CREWFAL  +  .16  GIVEGET  + 

(-.46)  WORKMNG  +  .69  XMNITOR  + 

.23  TEAMACE  +4.65  .72 

3. )  %OFFCOUR  =  (-8.58)  ATMALL  +  (-6.57)  TEAMCMAQ  + 

(-5.06)  HLPCMAQ  +  (-15.59)  INFOEXC  + 

9.36  CREWFAL  +  (-1.08)  GIVEGET  + 

(-13.92)  WORKMNG  +  9.96  XMNITOR  + 

7.54  TEAMCMAQ  +  105.16  .68 

4. )  WITHIN  =  (-.35)  ATMALL  +  (-.27)  TEAMCMAQ  + 

(-.14)  HLPCMAQ  +  .13  INFOEXC  + 

(-.03)  CREWFAL  +  .30  GIVEGET  + 

.19  WORKMNG  +  (-.15)  XMNITOR  + 

.31  TEAMACE  +  .06  .71 

5. )  THRT#  =  (-3.17)  ATMALL  +  .05  TEAMCMAQ  + 

.42  HLPCMAQ  +  .05  INFOEXC  + 

(-.43)  CREWFAL  +  (-.49)  GIVEGET  + 

(-1.54)  WORKMNG  +2.11  XMNITOR  + 

.54  TEAMACE  +9.97  .60 

6. )  THRTIME  =  (-44.44)  ATMALL  +  (-5.65)  TEAMCMAQ  + 

(-1.09)  HLPCMAQ  +  (-6.47)  INFOEXC  + 

2.91  CREWFAL  +  .37  GIVEGET  + 

(-27.32)  WORKMNG  +  29.16  XMNITOR  + 

12.01  TEAMACE  +  149.65  .59 

7. )  THRTMAX  =  (-15.02)  ATMALL  +  (-3.23)  TEAMCMAQ  + 

(-4.87)  HLPCMAQ  +  (-8.65)  INFOEXC  + 

7.74  CREWFAL  +3.46  GIVEGET  + 

(-3.65)  WORKMNG  +5.32  XMNITOR  + 

5.44  TEAMACE  +44.38  .59 

8. )  MEANDUR  =  (-2.99)  ATMALL  +  (-2.32)  TEAMCMAQ  + 

(-1.22)  HLPCMAQ  +  .35  INFOEXC  + 

2.48  CREWFAL  +  1.10  GIVEGET  + 

(-.87)  WORKMNG  +  .47  XMNITOR  + 

.10  TEAMACE  +19.18  .38 

9. )  ILSRIGHT  =  (-.63)  ATMALL  +6.78  CREWFAL  + 

5.17  HLPCMAQ  +  (-6.38)  WORKMNG  + 

2.18  TEAMCMAQ  +  (-10.44)  GIVEGET  + 

15,32  XMNITOR  +  (-9.17)  TEAMACE  + 

1.06  INFOEXC  +70.18  .71 


%  Variance 


50  *★* 


52  *** 


46  ** 


50  *** 


36 


35 


35 


14 


51  *★* 


+  Levels  of  significance  are: 


itir-k 


p  <  .01  **  p  <  .05,  and  *  p  <  .10. 


The  equations  developed  in  Table  6.8-1  demonstrate  that,  when  considered  together,  the  CMAQ 
scales,  ACE  Checklist  scales  and  ATMALL  have  considerable  predictive  power  in  determining 
performance  -  especially  navigation-related  performance.  By  entering  the  independent  variables 
in  a  stepwise  manner,  it  was  expected  that  the  better  predictors  would  become  apparent.  Table 
6.8-2  shows  these  results. 


Tabl*  6.8-2 

Stepwise  Regression  of  CMAQ  "Logical"  Scales, 
ACE  Checklist  Scales,  and  AOSJALL 
with  Performance  Variables* 

(n«40) 


Ecniation 

1  )  NAVTIME  =  (-3.48)  WORKMNG  +  (-2.62)  HLPCMAQ  + 
56.97 

2. )  DEVIATE#  *  (-.58)  WORKMNG  +3.41 

3. )  %OFFCOUR  =  (-12.68)  WORKMNG  +  68.40 

4!)  WITHIN  =  .44  TEAMACE  +  (-.42)  ATMALL  +  (-.31) 

5. )  THRT#  *  NE  ^  ^ 

6. )  THRTIME  *  (-31.25)  WORKMNG  +  21.20  XMNITOR  +  £ 

7. )  THRTMAX  =  (-12.19)  ATMALL  +  49.39 

8 .  )  MEANDUR  =  NE 

9. )  ILSRIGHT  =  13.36  XMNITOR  +  (-11.75)  TEAMACE  + 

82.52 

*  In  all  equations,  F  is  significant  at  the  p  <  .C 


Multiple  R 

%  Variance 

.68 

46 

.53 

28 

.54 

29 

.56 

32 

54  .51 

26 

.41 

17 

.58 

level. 

34 

In  these  equations,  it  can  be  seen  that  one  of  the  CMAQ  subscales  (HLPCMAQ),  three  of  the 
ACE  subscales  (WORKMNG,  XMNITOR,  and  TEAMACE),  and  the  ATMALL  measure 
significantly  contribute  to  at  least  one  simulator  mission  performance  variable.  Inspection  of  the 
computer  printouts  of  the  stepwise  entry  of  the  variables  revealed  that  in  most  equations  the 
ACE  accounted  for  the  predominant  amount  of  the  variance.  Of  particular  note  is  the  fact  that 
an  attitude  subscale  (HLPCMAQ)  entered  the  stepwise  regression  indicating  its  statistical 
significance.  This  finding  was  very  important  and  encouraged  further  analyses  to  better 
understand  the  relationship  between  attitude  and  performance. 


6.9  Predict  Performance  Variables  Using  ATMALL.  CMOALL.  .and  ACEALL 


The  ATMALL,  CMQALL,  and  ACEALL  measures  were  forced  into  regression  equations  with 
the  nine  simulator  performance  variables.  The  results  of  these  analyses  can  be  seen  in  Table 
6.9-1. 
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Table  6 • 9-1 

Forced  Regreeeion  of  ATMALL,  ACEAIi,  and  CMQALL 
with  Performance  Measures^ 

(n=40) 


Equation 

Multiple  R 

%  Variance 

1. ) 

NAVTIME  = 

(-2.52)  CMQALL  +  (-2.37)  ATMALL  + 
(-2.11)  ACEALL  +  56.56 
*  (-.08)  CMQALL  +  (-.76)  ATMALL  + 

.52 

27 

★  ★  ★ 

2.) 

DEVIATE# 

.54 

30 

if** 

(-.31)  ACEALL  +4.81 
«  (-6.61)  CMQALL  +  1.46  ATMALL  + 

3.) 

%OFFCOUR 

.49 

24 

** 

(-13.73)  ACEALL  +  104.87 
(-.01)  CMQALL  +  (-.44)  ATMALL  + 

4.) 

WITHIN  = 

.57 

32 

*** 

.48  ACEALL  +  (-.24) 

5.) 

THRT#  =  ( 

-1.51)  CMQALL  +  (-2.17)  ATMALL  + 

.33 

11 

93  ACEALL  +  14.46 

6.) 

THRTIME  = 

(-16.88)  CMQALL  +  (-29.48)  ATMALL  + 
4.70  ACEALL  +  200.45 

.31 

10 

7.) 

THRTMAX  = 

3.25  CMQALL  +  (-9; 64)  ATMALL  + 

(-2.05)  ACEALL  +  32.44 
t  .32  CMQALL  +  (-3.31)  ATMALL  + 

.44 

19 

* 

8.) 

MEANDUR  = 

.26 

9 

.43  ACEALL  +  16.68 

9* ) 

ILSRIGHT 

=2.26  CMQALL  +  2.70  ACEALL  + 

.28 

8 

6.81  ATMALL  +  45.19 

+  Levels  of  significance  are: 


<  .01  **  p  <  .05,  and  *  p  <  .10. 


Table  6.9-1  shows  that  the  ATMALL,  CMQALL,  and  ACEALL  measures,  taken  together,  m 
able  to  explain  a  significant  amount  of  the  variance  of  several  performance  measures,  especially 
those  variables  that  are  navigation-related. 

Stepwise  regressions  were  next  computed,  and  the  results  are  shown  in  Table  6.9-2. 


Tabl*  6.9-2 

Sbepwlse  Regression  of  ACEALL/  ATMALL/  and  CMQALL 
with  Performance  Measures* 

(n=40) 

_  Multiple  R 

Equation 

1. )  NAVTIME  =  (-3.06)  ACEALL  +40.42  -45 

2. )  DEVIATE#  =  (-1.15)  ATMALL  +4.24  • 

3. )  %OFFCOOR  =  (-13.04)  ACEALL  +  70.68 

4. )  WITHIN  =  .48  ACEALL  +  (-.44)  ATMALL  +  (-.28)  .57 

5 .  )  THRT#  =  NE 

6. )  THRTIME  =  NE 

7. )  THRTMAX  =  (-12.19)  ATMALL  +49.39 

8 .  )  MEANDUR  =  NE 

9 .  )  ILSRIGHT  =  NE 

*  In  all  equations,  F  is  significant  at  the  p  <  .01  level. 


^  Variance 

20 

27 

22 

32 


17 
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Table  6  9-2  indicates  that,  as  in  Paragraph  6.5,  the  CMQALL  scale  again  drops  from  all 
equations  when  stepwise  entry  is  used.  Inspection  of  the  result  shows  the  ATM^L  scale  to 
S  the  best  predictor  of  DEVIATE#  and  THRTMAX  while  The  ACEALL  scale  is  the  best 
predictor  of  NAVTIME,  %OFFCOUR,  and  WITHIN. 


6.10  Summary 

Several  of  the  equations  presented  in  this  Section  demonstrate  that  the  ACE  Checklist  scales  are 
powerful  predictors  of  ATM  Task  performance.  Likewise,  ACE  scales  are  also  good  predictors 
of  the  simulator  performance  measures. 

ATMALL  proved  to  be  a  powerful  predictor  of  performance.  This  finding  was  encouraging  since 
ATM  Task  performance  is  a  central  component  of  the  APART  program  and  lends  credibility  to 
the  APART  reliance  on  the  ATM  Tasks. 


The  performance  of  the  CMAQ  "logical"  scales  when  using  a  sample  size  of  40  w^^dly 
encouraging.  Analysis  indicated  some  relationship  between  attitudes  (the  Army  CMAQ)  and 
behavior  (as  rated  on  the  ACE  and  the  GradesUps)  and  mission  performance  in  the  simulator. 
In  short,  the  attitude  behavior/performance  linkage  was  established,  albeit  a  weak  one. 


Section  7  will  focus  on  the  question  of  how  well  the  CMAQ  "factor"  scales  perform  in  depicting 
the  attitude  -♦  behavior/performance  relationship. 
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Section  7.0 

'Relationships  Amon^  the  Measures:  CMAQ  "Factor"  ScalgS- 
7.1  Introduction 

The  results  presented  in  the  previous  Section  demonstrate  the  strong  statistical  relationships 
among  the  ACE  CheckHst,  ATM  Tasks,  and  simulator  performance  variables.  The  measures 
were  shown  to  have  performed  as  expected.  Since  the  relationship  between  attitudes  (the  Army 
CMAQ  "logical"  scales)  and  the  other  meaures  was  less  than  postulated,  alternative  ways  of 
analyzing  the  data  were  sought.  The  next  Sections  present  various  explorations  of  the  CMAQ 
data,  and  present  several  interesting  conclusions. 

Gregorich  et  al.  (1990)  used  a  factor  analytic  model  to  develop  the  CMAQ  scales  used  in  the 
NASA/UT  analyses.  As  discussed  in  Paragraph  2.5,  the  Army  CMAQ  revealed  three  factors 
very  siimlar  to  those  of  the  NASA/UT  analysis.  Thus,  it  was  decided  to  use  the  three  factors 
derived  from  the  Army  CMAQ  factor  analysis  and  recompute  certain  of  the  equations  found  in 
Section  6.  This  Section  presents  the  results. 

The  variable  names  used  in  this  Section,  including  the  Army  CMAQ  "factor"  scale  names,  were 
listed  previously  in  Table  6.1-1.  The  organizational  chart  for  this  Section  is  at  Table  7.1  1. 
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Tabl«  7.1-1 

Organxza'ti.onal  Chairt  for  Section  7 


Analysis/Equation 

I  nt  erpret  at  ion 

Table (s) 

7.2  Predict  ATM 
performance  using  CMAQ 
"factor"  scales 

CMAQ  "factor"  scales 
are  not  predictive  of 

ATM  performance. 

7.3  Predict  ACE 
performance  using  CMAQ 
"factor"  scales 

CMAQ  "factor"  scales 
are  not  predictive  of 

ACE  performance. 

7.3-1 

7.4  Predict  ATM 
performance  using  CMAQ 
"factor"  scales  and  ACE 
scales 

More  variance  explained 
than  either  independent 
variable  alone.  CMAQ 
"factor"  scales  drop 
from  the  stepwise 
equations. 

7.4-1 

7.5  Predict  ATM 
performance  using 

ACE ALL  and  CMAQ34 

High  predictive  value 
when* taken  together; 
CMAQ34  drops  from  the 
equation  when  scales 
are  entered  stepwise. 

7.6  Predict  performance 
variables  using  CMAQ 
"factor"  scales 

CMAQ  "factor"  scales 
have  some  predictive 
value. 

7.6-1,  7.6-2 

7.7  Predict  performance 
variables  using  CMAQ 
"factor"  scales,  ACE 
scales,  and  ATMALL 

Mission  performance  can 
be  predicted*  At  least 
one  scale  from  each  of 
the  three  instruments 
enters  the  stepwise 
equations . 

7.7-1,  7.7-2 

7.8  Predict  performance 
variables  using  ATMALL, 
CMAQ34,  and  ACE ALL 

The  three  measures  have 
moderate  predictive 
power;  CMAQ34  slightly 
improves  prediction, 
but  drops  from  the 
stepwise  equation. 

7.8-1 
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7  2  Predict  ATM  Performance  Using  CMAO  "Factor — $c^g§ 

The  three  factor  scales^  (COMMCOR,  SHARLEAD,  STRESS)  were  forced  into  equations  with 
the  five  ATM  scales.  Table  7.2-1  shows  the  resultant  data. 


Tabl*  7.2-1 

Forced  Regreseion*  of  CMAQ  "Factor"  Scales 
with  ATM  Measures* 


(n*40) 

Ecruation 

Multiple  R 

%  Variance 

1.) 

ATMALL  *  ( 
( 

-.07)  STRESS  +  .11  COMMCOR  + 

-.01)  SHARLEAD  +2.05 

.16 

3 

2.) 

ATM_13  «  ( 

-.01)  STRESS  +  (-.02)  COMMCOR  + 

.04  SHARLEAD  +2.41 

.05 

0 

3.) 

ATM_12  = 

(-.02)  STRESS  +  (t.03)  COMMCOR  + 

.03  SHARLEAD  +2.58 

.05 

0 

4.) 

BIGRADE  = 

(-.02)  STRESS  +  .21  COMMCOR  + 

(-.10)  SHARLEAD  +  1.58 

.11 

1 

5.) 

TASK1071  = 

:  .10  STRESS  +  .10  COMMCOR  + 

.06  SHARLEAD  +  .83 

.15 

2 

*  In  all  equations,  F  is  not  significant  at  the  p  <  .05  level. 


This  table  can  be  compared  to  Table  6.3-1,  which  reports  the  results  of  similar  analyses 
performed  with  the  CMAQ  "logical"  scales.  Although  the  "factor"  scales  have  higher  reUabilities 
than  the  "logical"  scales,  the  CMAQ  still  has  insignificant  predictive  value  in  determining  ATM 
performance.  Comparison  by  inspection  of  Tables  6.3-1  and  7.2-1  indicates  that  the  logical 
scales  are  slightly  better  in  accounting  for  variance  in  ATM  Task  performance.  Since  F  was  not 
significant,  stepwise  regression  equations  were  not  computed. 

7^  Prpdirt  ACE  Performance  Using  CMAO  "Factor"  Scales 

Next,  the  three  CMAQ  "factor"  scales  were  forced  into  equations  with  the  six  ACE  subscales. 
The  results,  shown  in  Table  7.3-1,  are  similar  to  those  presented  in  Table  6.4-1.  The  CMAQ, 
in  either  its  "logical"  or  "factor"  form,  has  little  predictive  value  when  regressed  with  the  ACE 
measures.  A  possible  explanation  for  this  finding  is  presented  in  Paragraph  7.9. 


^  Note:  CMAQ34  was  not  included  in  the  regression  equations  because  its  use  would 
introduce  interdependency  among  the  predictor  variables.  It  will  be  entered  singly,  or  in 
conjunction  with  other  overall  scales,  in  later  analyses. 
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Tabl«  7.3-1 


Forced  Regression  of  CMAQ  '•Factor** 
with  ACE  Measures* 

(n=40) 

Scales 

Ecruation 

Multiple  R 

%  Variance 

1.) 

ACEALL  « 

(.01)  STRESS  +  .26  COMMCOR  + 

(-.13)  SHARLEAD  +  2.49 

.14 

2 

2.) 

TEAMACE 

=  (-.08)  STRESS  +  .02  COMMCOR  + 

,03  SHARLEAD  +3.66 

.09 

1 

3.) 

XMNITOR 

*  (-.02)  STRESS  +  .21  COMMCOR  + 

(-.18)  SHARLEAD  +  3.15 

.12 

2 

4.) 

INFOEXC 

=  .04  STRESS  +  .33  COMMCOR  + 

(-.29)  SHARLEAD  +2.66 

.24 

6 

5.) 

WORKMNG 

=  (-.002)  STRESS  +  ;43  COMMCOR  + 

(-.01)  SHARLEAD  +  .75 

.22 

5 

6.) 

GLOBAL  »  (-.06)  STRESS  +  .38  COMMCOR  + 

(-.10)  SHARLEAD  +  1.87 

.15 

•  2 

*  In  all  equations,  F  is^not  significant  at  the  p  <  .05  level 


7.4  Predict  ATM  Performance  Using  CMAO  "Factor*  Scales  and  ACE.S£^.£S 

The  three  CMAQ  "factor"  scales  and  four  ACE  subscales  were  then  forced  into  equations  with 
the  ATM  Task  subscales  as  the  dependent  variable.  Table  7.4-1  shows  the  results  of  the  forced 

regression. 
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Forced 


Table  7 .4-1 

Regression  of  CMAQ  "Factor"  Scales  and  ACE  Scales 
with  ATM  Measures* 

(n“40 ) 


1.  ) 


Equation 


ATMALL  = 


.10  WORKMNG  +  (-.04)  STRESS  + 
(-.01)  SHARLEAD  +  (.06)  COMMCOR  + 
.18  XMNITOR  +  (-.12)  INFOEXC  + 

.23  TEAMACE  +  .91 


Multiple  R 


.78 


2.)  ATM_13 


.23  WORKMNG  +  .02  STRESS  + 

.02  SHARLEAD  +  (-.10)  COMMCOR  + 
.21  XMNITOR  +  (-.19)  INFOEXC  + 
.19  TEAMACE  +  1.37 


.82 


3 . )  ATM_12 


.28  WORKMNG  +  .01  STRESS  + 

(-.01)  SHARLEAD  +  (-.11)  COMMCOR  + 
.21  XMNITOR  +  (-.26)  INFOEXC  + 

.17  TEAMACE  +  1.81 


.79 


4.) 


BIGRADE  = 


.51  WORKMNG  +  (-.004)  STRESS  + 
(-.09)  SHARLEAD  +  (-.02)  COMMCOR  + 
.31  XMNITOR  +  (-.17)  INFOEXC  + 

.003  TEAMACE  +  .66 


.76 


5.) 


TASK1071 


»  (-.24)  WORKMNG  +  .12  STRESS  + 
.24  SHARLEAD  +  (-.03)  COMMCOR  + 
.24  XMNITOR  +  .53  INFOEXC  + 

.44  TEAMACE  +  (-2.77) 


.79 


*  In  all  equations,  F  is  significant  at  the  p  < 


.0001  level. 


%  Variance 


61 


67 


63 


59 


63 


The  results  in  terms  of  the  percent  of  variance  explained  in  Table  7.4-1,  are  nearly  identic^ 
» those  priented  in  Table  6.5-1.  When  the  data  are 

manner,  the  results  are  the  same  as  those  shown  m  Table  6.5-2;  i.e.,  the  CMAQ  scales  d  p 
from  the  equation. 


7.5  Predict  AHTM  Performance  Using  ACEALL  and  CMAQM 

The  CMAQ34  and  ACEALL  scales  were  forced  into  equations  wiA  the 

These  equations  are  at  Table  7.5-1  and  are  nearly  idenUcal  to  those  at  Table  6.5-3.  men 

anowedT  enter  stepwise,  only  ACEALL  enters  and  the  equations  are  the  same  as  those  in  Table 

6.2-3. 
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Table  7.5-1 

Forced  Regression  of  CMAQ34  and  ACEALL  Scales 
with  ATM  Measures* 

( n*40 ) 

Multiple  R 


%  Variance 


1. )  ATMALL  =  (-.02)  CMAQ34  +  .42  ACEALL  +  1.13  -71 

2. )  ATM  13  *  .01  CMAQ34  +  .47  ACEALL  +  .85  .72 

3  )  ATM"12  =  (-.02)  CMAQ34  +  .43  ACEALL  +  1.18  .66 

4!)  BIGRADE  *  (-.01)  CMAQ34  +  .73  ACEALL  +  (-.16)  .69 

5.)  TASK1071  =  .25  CMAQ34  +  .94  ACEALL  +  (-2.32)  .74 

*  In  all  equations,  F  is  significant  at  the  p  <  .0001  level. 


The  CMAQ  "factor"  scales  were  placed  into  regression  equations  with  the  mission  performance 
variables  as  the  dependent  measure.  Tables  7.6-1  and  7.6-2,  respectively,  show  the  results  of 
both  the  forced  and  stepwise  entry  of  the  CMAQ  "factor"  scales  into  the  regression  equations. 


lation 


Tabla  7,6-1 

Forced  Regression  of  CMAQ  "Factor”  Scales 
with  Performance  Measures^ 

(n*40) 

Multiple  R 


1.  )  NAVTIME  =  : 

2 .  )  DEVIATE#  = 

3 .  )  %OFFCOUR  = 

4. )  WITHIN  =  . 

5. )  THRT#  =  (- 

(- 

6. )  THRTIME  = 

7. )  THRTMAX  = 

8.  )  MEANDUR  = 

9 .  )  ILSRIGHT  = 


1.12  STRESS  +  (-3.17)  COMMCOR  + 
(-1.51)  SHARLEAD  +  52.97 
=  .19  STRESS  +  (-.66)  COMMCOR  + 

.21  SHARLEAD  +3.42 
=  2.41  STRESS  +  (-24.96)  COMMCOR  + 
6.36  SHARLEAD  +  128.61 
.02  STRESS  +  (-.02)  COMMCOR  + 

.004  SHARLEAD  +  .26 

-.39)  STRESS  +  .35  COMMCOR  + 

-.88)  SHARLEAD  +8.87 
(-3.87)  STRESS  +  (-2.02)  COMMCOR  + 
(-8.76)  SHARLEAD  +  133.51 
.83  STRESS  +  (-4.59)  COMMCOR  + 

2.54  SHARLEAD  +  29.66 

.29  STRESS  +  (-1.61)  COMMCOR  + 

.21  SHARLEAD  +  19.12 
=  1.00  STRESS  +  (-.47)  COMMCOR  + 
(-1.28)  SHARLEAD  +  89.08 


%  Variance 


20 


p  <  .01,  **p  <  .05,  and  *  p  <  .10, 


+  Levels  of  significance  are: 


Table  7 • 6-2 

Stepwise  Regression  of  CMAQ  "Factor 
with  Perfoirmance  Measures* 

(n»=40) 


Equation 


Scales 


Multiple  R 


1. )  NAVTIME  =  (-3.74)  COMMCOR  +  52.59 

2. )  DEVIATE#  =  NE 

3. )  %OFFCOUR  =  (-17.70)  COMMCOR  +  132.40 

4 .  )  WITHIN  =  NE 

5 .  )  THRT#  =  NE 

6 .  )  THRTIME  *  NE 

7 .  )  THRTMAX  -  NE 

8 .  )  MEANDUR  »  NE 

9 .  )  ILSRIGHT  «  NE 

*  In  all  equations,  F  is  significant  at  the  p  <  .05  level 


%  Variance 
11 
15 


A  note  of  explanation  is  offered  at  this  point.  Previously,  in  Table  6.7-1,  equations  were  built 

using  the  C^Q  "logical"  scales;  however,  since  aI  So 

stepwise  regressions  could  not  be  computed.  As  shown  m  Tables  ^6-1  and  7.6-2  the  Ch^Q 

"factor"  scies  have  predictive  value  when  regressed  wiA 

Table  7  6-1  shows  that  significant  results  were  obtained  when  the  CMAQ  factor  scales  were 
used  in  the  equations.  This  finding  made  it  possible  to  develop  the  ^nations  present^ 

7.6-2.  Again,  Tables  7.6-1  and  7.6-2  give  evidence  to  the  Imkage  between  attitudes  an 

performance  as  noted  earlier  in  Paragraph  6. 10. 

^  e  the  C 

To  determine  the  net  effect  of  the  measurement  suite  on  performance,  the  three  CMAQ  "factor" 
^“four  ACE  subscales,  and  the  ATMALL  measure  were  for^  mto  regresston 
equations  with  the  nine  performance  vanables.  The  results  are  at  Table  7.7-1. 


Table  7.7-1 

Forced  Regression  Using  CMAQ  "Factor"  Scales, 
ACE  Checklist  Scales,  and  ATMALL 
with  Performance  Variables+ 

(n*40) 


Equation 


Multiple  R 


%  Variance 


1 .  )  NAVTIME  * 

2 .  )  DEVIATE#  ' 

3 .  )  %OFFCOUR  ' 

4. )  WITHIN  = 

5. )  THRT#  = 

6. )  THRTIME  = 

7. )  THRTMAX  > 

8 .  )  MEANDUR  = 

9. )  ILSRIGHT 


(-2.11)  ATMALL  +  (-.89)  SHARLEAD  + 

.94  STRESS  +  (-2.20)  COMMCOR  + 

.09  INFOEXC  +  (-3.42)  WORKMNG  + 

2.31  XMNITOR  +  (-1.40)  TEAMACE  + 

57.66 

e  (-1.22)  ATMALL  +  .14  SHARLEAD  + 

.17  STRESS  +  (-.31)  COMMCOR  + 

(-.60)  INFOEXC  +  (-.42)  WORKMNG  + 

.66  XMNITOR  +  .15  TEAMACE  +  5.25 
=  (-5.66)  ATMALL  +  4.04  SHRALEAD  + 

2.96  STRESS  +  (-17.28)  COMMCOR  + 
(-11.07)  INFOEXC+  (-11.30)  WORKMNG  + 
8.32  XMNITOR  +3.14  TEAMACE  +  140.50 
(-.41)  ATMALL  +  .02  SHARLEAD  + 

.001  STRESS  +  (-.09)  COMMCOR  + 

.16  INFOEXC  +  .20  WORKMNG  + 

(-.14)  XMNITOR  +  .28  TEAMACE  +  (-.08) 
(-3.37)  ATMALL  +  (-.50)  SHARLEAD  + 
(-.48)  STRESS  +  .85  COMMCOR  + 

(-.25)  INFOEXC  +  (-1.57)  WORKMNG  + 
2.19  XMNITOR  +  .74  TEAMACE  +  8.15 
(-46.48)  ATMALL  +  (-5.58)  SHARLEAD  + 
/_4.77)  STRESS  +  9.16  COMMCOR  + 
(-7.78)  INFOEXC  +  (-26.52)  WORKMNG  + 
28.67  XMNITOR  +  13.34  TEAMACE  + 
131.96 

(-14.34)  ATMALL  +  .99  SHARLEAD  + 

.68  STRESS  +  (-.66)  COMMCOR  + 

(-6.34)  INFOEXC  +  (-2.62)  WORKMNG  + 
3.88  XMNITOR  +4.38  TEAMACE  +  49.90 
:  (-2.81)  ATMALL  +  .72  SHARLEAD  + 

.02  STRESS  +  (-1.79)  COMMCOR  + 

1.23  INFOEXC  +  (-.26)  WORKMNG  + 

.08  XMNITOR  +  (-.69)  TEAMACE  +  24.22 
=  3.15  ATMALL  +  (-1.38)  COMMCOR  + 

1.04  STRESS  +  2.34  INFOEXC  + 

(-.33)  SHARLEAD  +  (-6.92)  WORKMNG  + 

14.44  XMNITOR  +  (-11.58)  TEAMACE  + 

92.44 


.74 


.74 


.70 


.65 


.62 


.61 


.51 


,31 


.62 


54  *** 


54 


49  *** 


42  ** 


39  ** 


37  * 


26 


38 


+  Levels  of  significance  are:  ***  p  <  *01,  **  p  <  .05,  and  *  p  <  .10. 


Table  7  7-1  can  be  compared  to  Table  6.8-1 .  On  inspection,  both  tables  have  simil^  percentoges 
of  variance  explained,  but  the  use  of  the  CMAQ  "factor"  scales  results  m  a 
statistically  significant  equations  being  developed.  This  finding  lends 

notion  that  the  "factor"  scales  are  better  than  the  "logical"  scales  m  prediction  of  performance. 
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Next,  by  allowing  the  independent  variables  to  enter  the  equation  in  a  stepwise  manner,  it  was 
expected  that  the  more  predictive  variables  would  be  revealed.  Table  7.7-2  shows  the  results  of 
these  calculations. 


Ecmation 


Tabl*  7 •7-2 

Regression  of  CMAQ  ‘‘Factor*’ 
S  Checklist  Scales,  and  AXMi 
with  Performance  Measures* 
(n*40) 


1. )  NAVTIME  =  (-3.53)  WORKMNG  +  41.80 

2. )  DEVIATE#  *  (-.58)  WORKMNG  +3.41 

3. )  %OFFCOUR  *  (-11.24)  WORKMNG  +  (-12.93)  COMMCOR  + 

140.69 

4. )  WITHIN  =  .44  TEAMACE  +  (-.42)  ATMALL  + 

(-.31) 

5 .  )  THRT#  =  NE 

6. )  THRTIME  =  (-31.25)  WORKMNG  +  21.20  XMNITOR  + 

86.54 

7. )  THRTMAX  =  (-12.19)  ATMALL  +  49.39 

8 .  )  MEANDUR  =  NE 

9. )  ILSRIGHT  =  13.36  XMNITOR  +  (-11.75)  TEAMACE  + 

82.52 


Scales , 

IT.T. 

Multiple  R 

%  Variance 

.61 

37 

.53 

28 

.61 

37 

.56 

32 

.51 

26 

.41 

17 

CO 

in 

• 

33 

*  In  all  equations,  F  is  significant  at  the  p  <  .01  level. 


Table  7.7-2  demonstrates  that  the  CMAQ  "factor"  scales  have  some  predictive  power. 
Specifically,  the  COMMCOR  factor  becomes  an  important  component  in  the  prediction  equation 
of  the  %OFFCOUR.  This  result  is  similar  to  that  reported  in  Table  6.8-2,  where  a  CMAQ 
"logical"  scale  (HLPCMAQ)  appeared  in  a  stepwise  equation  predicting  another  navigation- 
related  variable  (NAVTIME).  Table  2.5-3  suggests  a  linkage  between  the  "logical"  scale  entitled 
"Provide/ Accept  Help"  (HLPCMAQ)  and  the  "factor"  scale  entitled  "Communication  & 
Coordination"  (COMMCOR).  Interestingly,  it  is  the  HLPCMAQ  scale  in  Table  6.8-2  and  the 
COMMCOR  scale  in  Table  7.7-2  that  enter  the  stepwise  equations  as  significant  predictors  of 
performance.  Thus,  there  is  corroborating  evidence  that  1)  attitudes  impact  navigation-related 
performance,  and  2)  the  HLPCMAQ  and  COMMCOR  scales  are  related. 

2A  Predict  Performance  Variables  Using  ATMALL.  CMA034.  and  ACEALL 

In  this  analysis,  the  CMAQ34,  ACEALL,  and  ATMALL  were  forced  into  regression  equations 
as  the  independent  variables  with  the  performance  measures  as  the  dependent  variable.  Table 
7.8-1  shows  the  results  of  these  equations. 
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Xabls  7 .8”1 

«-F  rMAn3A  ACEAIiL,  and  ATMAIiL 
Forced  Regression  of  cMAa-**' 

Force  s  Performance  Measures+ 

(n=40) 


Bouation 

1. )  SAVTIME  -  (-2.66)  CJttaM  *  * 

'  J-1.98)  ACEALL  +  57.71 

2. )  DEVIATE#  =  .01  CMAQ34  +  (-.77)  ATMALL  + 

'  (-.31)  ACEALL  +  4.31 

3  )  %0FFC0UR  -  (-7.65)  CMAQ34  +1*03  ATMALL  + 
(-13.37)  ACEALL  +  111.64 
4.)  WITHIN  «  (.001)  C^Q34  +  (-.44)  ATMALL  + 

.o  TH.«  .  ^ 

6.)  THRTIME  CMAQji  .  (-30.«)  ATMMX  . 

'  5.54  ACEALL  +  206.37 

7  )  THRTMAX  =  .67  CMAQ34  +  ^ 

’  (-2.13)  ACEALL  +  46.44  ^ 

8. )  MEANDUR  »  (-.62)  CMAQ34  +  ATMAL 

'  .44  ACEALL  +  21.93 

9. )  ILSRIGHT  «  .58  CMAQ34  +  2-60  ACEALL  + 

'  e.  fift  ATMALL  +  54 . 89 


Mil  1  + role  R  *  Variance 


.54 

.54 

.51 

.57 

.34 

.33 

.42 

.27 


29  *** 

30  *** 
26  *** 
32  *** 
11 

11 

18  * 

7 

8 


However,  CMAQ34  consistently  dropped  froin  the  ^^tions,  tn 
computations  are  identical  to  those  found  m  Table  6.9-2. 

7.9  Summary 

Se«ions  6  and  7  have  shown  that  fte  ACE  Chechlis.  -d  ATM  ^s  «Mbi. « 
Checklist  or  ATM  Task  measures,  however  it  was  very  mteres  g  performance 

~  semes^^  -.hd, 

better  predictors  than  the  logical  scales. 

EstabUshment  of  d.e  "behes  p^dicdve'  JS  to 

performance  may  be  due  in  part  to  the  compara  ve  y  "lo«^ical"  scales-  and  the  mission 

die  CMAQ  "factor"  scales  are  more  reliable  than  the  CMAQ  logical  scales,  ana 
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performance  measures  are  more  reliable  than  the  ACE  Checklist  or  ATM  Tasks.  Consequently, 
there  is  less  "noise"  (or,  unreliability)  in  the  correlations  between  the  CMAQ  "factor"  sc^es  and 
the  mission  performance  variables.  Thus,  with  less  "noise"  present  as  an  artifact  of  the 
measures,  the  true  relationship  can  be  established.  This  is  not  to  say  that  the  ACE  and  ATM 
measures  are  unreliable;  rather,  it  indicates  that  the  mission  performance  measures  as  collected 
in  the  simulator  are  relatively  "objective"  while  the  ACE  Checklist  and  ATM  Task  measures, 
by  definition,  are  subjective  ratings;  therefore,  they  are  exposed  to  more  "noise"  in  the  data  than 
the  performance  variables. 

As  a  result  of  the  analyses  explained  in  Sections  6  and  7,  a  relationship  was  empirically 
established  between  attitudes  towards  crew  coordination  and  performance.  Based  on  DRC’s 
literature  review  accomplished  to  date,  this  is  the  first  time  such  a  relationship  has  been  shown 
empirically.  While  others  have  postulated  or  assumed  the  existence  of  such  a  relationship,  none 
have  empirically  proved  it.  The  fact  that  a  relationship  between  attitudes  and  performance  was 
established  necessitated  further  analyses  to  better  understand  the  nature  of  the  relationship. 
Sections  8  and  9  present  the  analyses  and  results  of  this  in-depth  examination. 
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.Sgctipn  giL  .  o  1  . 

T?piatinn^hins  Among  the  M^^^nres  Using  the  CMAQ  "Logical  Scales. 

virions  Crpw  r.ombinations 


S.l  Introduction 

Tn  Sections  6  and  7,  the  results  of  regression  equations  with  CMAQ,  AM,  ACE, 
Wo^a  measure;  were  presented.  Several  findings  affirmed  that  the  CMAQ  scales  h^ 
ffightly  better  predictive  ability  when  the  dependent  variables  were  j^rfonnance  m^u^, 
litfie  if  any,  predictive  value  when  regressed  against  ATM  or  ACE  scales.  As 
Son  7  reiLary,  the  performance  variables  being  objective  measures  are  subject  to  less 
IL  t^™  of  ACE  measures,  and  this  could  account  for  the  better  regression 

equations  in  those  cases. 

It  was  determined  that  using  the  40  CMAQ  attitude  observations  to 

performance,  when  only  20  behavior/performance  observations  existed,  was  probably  nm  Ae 

Lst  approach.  Multiple  regression  is  a  linear  manipulation;  but,  by  attempting  to  relate  CMAQ 

aSS  which  are'^hasel  on  the  individual  (n=40),  .^th  ATM  ACE  -f  r 

variables  which  are  based  on  the  crew  (n=20),  a  bias  m  the  view  of  the  attitude  - 

behavior/performance  relationship  is  introduced.  An  example  of  the  consequences  is 

tiie  CMAQ  scales  are  regressed  with  the  ATM  scales,  40  ATM  cases  are  considered  ag^st  40 

CMAQ  cases.  The  reality  is  that  only  20  ATM  observations  are  available.  In  the  equations  of 

Sections  6  and  7,  this  complication  (40  observations  onto  20  observations)  is  present  for  any 

instance  when  the  CMAQ  scales  are  used. 

Accordinslv  for  both  statistical  and  rational  reasons,  it  was  postulated  that  a  con^inati^  of 
attitudes  wito  an  aircrew  exists  and  that  some  combination  of  Pilot-in-Command  (PC)  and  Pilot 
(PI)  attitudes  may  better  depict  the  relationship  between  attitudes  and  actions.  Thus,  Sections  8 
^d  fwm  addrL  the  issue  of  whether  a  combination  of  PC-PI  attitudes  better  predicts 
behavior/performance  than  the  PC  and  PI  considered  mdependently.  Also 
combination  of  PC-PI  attitudes  that  account  for  the  most  variance  m  predicting 

section  8  incorporates  the  CMAQ  "logical-  scales  in  the  analyses; 

Section  9  uses  the  CMAQ  "factor"  scales  in  a  similar  set  of  analyses. 

To  accompHsh  the  analyses,  an  alteration  of  the  testbed  datable  was 

case  database,  a  20-case  database  was  developed.  This  20-case  database  differentiated  the  C  Q 
scores  of  the  PC  from  the  PI  within  one  record;  treating  the  aircrew  as  one  case^  It  w^  tiien 
possible  to  have  a  one-to-one  observation  ratio.  The  next  step  was  to  determine  which  elements 
of  this  database  most  influenced  performance. 

It  was  initially  hypothesized  that  certain  elements  of  the  research^ta  might  have  a  grater 
influence  than  others  on  the  relationships  between  attitudes  and  performance.  For  ™pte,  it 
could  be  that  only  the  attitude  of  the  PC  was  the  key  driver  of  good  performance  in  a  situation 
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where  the  PC  had  a  "good"  attitude,  the  PI  had  a  "bad"  attitude,  and  the  crew  received  high 
ACE  and  ATM  ratings.  This  line  of  inquiry  gave  rise  to  many  similar  questions.  To  answer 
them,  10  combinations  of  PC-PI  CMAQ  scores  were  created.  The  variable  names  and 
descriptions  of  the  combinations  are  presented  in  Table  8.1-1. 


Xabls  8.1-1 

Aircrew  (PC-PI)  CMAQ  Combination  Scores 

Variable  Name  Description  and  Formula 


PCONLY 
PIONLY 
PCANDPI 
DHL  PC 
ABSDIF 

REALDIF 

AD_BAD 

AD_GOOD 

DBL_BAD 

DBL  GOOD 


The  PC  score  only. 

The  PI  score  only. 

The  total  of  the  PC  and  PI  score.  (PC  +  PI) 

Two  times  the  PC  plus  PI.  ((2  *  PC)  +  PI) 

The  absolute  value  of  the  difference  between  the  PC  and  PI. 

I  PC  -  PIl 

The  difference  of  the  PC  minus  PI.  (PC-PI). 

Only  the  "bad*  attitude  in  the  cockpit,  i.e.,  the  lower  score 
of  either  the  PC  or  PI. 

Only  the  "good"  attitude  in  the  cockpit,  i.e.,  the  higher 
score  of  either  the  PC  or  PI. 

Two  times  the  "bad"  attitude  plus  the  "good"  attitude.  ((2  * 
BADpcorPi)  +  (1  *  GOODpc^Pc)) 

Two  times  the  "good"  attitude  plus  the  "bad”  attitude.  ((2  * 
GOODpcorPi)  *  BADpc^pj)) 


The  CMAQ  "logical"  scales  comprise  five  attribute  scales:  TEAMCMAQ,  CREWFAL, 
GIVEGET,  HLPCMAQ,  and  CMQALL.  These  five  scales  were  weighted  by  the  ten  different 
CMAQ  combination  scores  described  in  Table  8.1-1  and  correlated  with  the  ATM,  ACE,  and 
performance  measures.  The  complete  bivariate  correlation  matrix  is  in  Appendix  E.  Likewise, 
the  CMAQ  "factor"  scales  comprise  four  attribute  scales:  COMCORR,  SHARLEAD,  STRESS, 
and  CMAQ34.  The  four  "factor"  scales  were  weighted  by  the  ten  different  CMAQ  combination 
scores  and  correlated  with  the  ATM,  ACE  and  performance  measures;  the  correlation  matrix 
for  which  is  presented  at  Appendix  F. 

Clearly,  it  was  not  feasible  to  perform  all  subsequent  analyses  using  all  10  aircrew  attitude 
combination  scores.  Examination  of  the  correlation  matrices  provided  valuable  insights  as  to  how 
to  efficiently  approach  the  data  analysis.  Of  the  ten  possible  CMAQ  weights,  three  in  particular 
consistently  resulted  in  higher  correlations:  PCONLY,  PCANDPI,  and  ABSDIF.  Thus,  only 
these  three  combination  scores  were  included  in  subsequent  analyses. 

A  salient  modification  to  be  noted  between  Sections  6  &  7  and  Sections  8  &  9  is  that  in  the  latter 
two  Sections,  the  probabilities  of  F-to-enter  and  F-to-remove  fi'om  any  regression  equation  were 
relaxed  to  the  p  <  .15  and  p  <  .16,  respectively.  In  other  words,  instead  of  the  probability  of 
F-to-enter  being  .05,  as  in  the  previous  analyses  (Sections  6  &  7),  it  was  increased  so  that  the 
F-to-enter  probability  had  to  be  only  .15  or  less  (Sections  8  &  9).  The  relaxed  criteria  permitted 
more  opportunities  to  observe  how  the  CMAQ  scales  functioned  in  stepwise  regression 
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eauations  Since  a  quantitative  Unk  had  already  been  demonstrated  in  Sections  6  &  7,  and 
Scause  the  sample  size  is  small  and  the  nature  of  this  research  is  exploratory,  this  modification 

seemed  reasonable. 

Regression  equations  were  computed  using  ATM,  ACE,  and  performance  measures  as  dependent 
variables  and  CMAQ  scales  as  independent  variables  in  the  form  of  the  three  different  weights: 
PCONLY  PCANDPI,  and  ABSDIF.  Since  the  objective  of  this  research  thrust  was  to  determme 
the  ’'best"  'cMAQ  predictor  combination,  only  the  variable  entered  on  the  first  step  is  presented 
in  the  following  tables.  Furthermore,  if  more  than  one  predictor  combination  were  entered  mto 
an  equation,  the  resulting  Beta  weights  would  have  been  uninterpretable  smce  the  mdependent 
variables  would  have  been  dependent  upon  one  another. 


In  this  Section  analyses  using  the  Army  CMAQ  "logical"  scales  are  presented;  Section  9  uses 
the  Army  CMAQ  "factor"  scales.  Table  8.1-2  is  the  organizational  chart  for  the  remamder  of 

this  Section. 


Tabl«  8.1-2 

Or^axiiz&'tl.onftX  Chart  for  Sactioix  8. 
Amy  CMAQ  "Logical"  Scales 


Analy  s  is  /Ecjuat  ion 

Interpret  at ion 

Table(6) 

8*2  Crew  attitude 
combinations  of 

TEAMCMAQ  Scale  to 
predict  behavior/ 
performance 

No  predictive  value  in 
any  of  the  three 
weights  across  all 
performance  measures 

None 

8.3  Crew  attitude 
combinations  of  CREWFAL 
Scale  to  predict 
behavior /performance 

ABSDIF  predicts 

TASK1071;  PCONLY 
negatively  predicts 
THRTMAX 

None  (in  text) 

8.4  Crew  attitude 
combinations  of  HLPCMAQ 
Scale  to  predict 
behavior /performance 

ABSDIF  consistently 
predicts  ATM  Task 
measures ;  PCONLY  or 
ABSDIF  predict  several 

ACE  measures;  PCANDPI 
and  ABSDIF  predict 
several  performance 
variables. 

8.4-1,  8.4-2,  8.4-3 

8.5  Crew  attitude 
combinations  of  GIVEGET 
Scale  to  predict 
behavior /performance 

ABSDIF  predicts  three 
of  the  five  ATM  Task 
measures;  ABSDIF 
predicts  XMNITOR; 

PCONLY  or  ABSDIF 
predict  several 
performance  variables. 

8.5- 1,  in  text,  and 

8.5- 2 

8.6  Crew  attitude 
combinations  of  CMQALL 
Scale  to  predict 
behavior /performance 

ABSDIF  predicts 

BIGRADE,  XMNITOR  and 
ILSRIGHT;  PCONLY 
predicts  THRT#. 

In  text  and  8.6-1 

69 


L2  Crew  Attitude  Combinations  of  TEAMCMAO  Scale  to  Predict  Behavior/Pp.rfnrmi^npp 


The  three  weights  (PCONLY,  PCANDPI,  ABSDIF)  of  the  TEAMCMAQ  scale  were  regressed 
with  the  ATM  measures,  the  ACE  subscales,  and  the  performance  variables.  Despite  the  relaxed 
criteria  of  F-to-enter  and  F-to-remove,  none  of  the  TEAMCMAQ  weightings  entered  into  any 
regression  equation. 

L2  Crew  Attitude  Combinations  of  CREWFAL  Scale  to  Predict  Behavior/Performanrp 

The  CREWFAL  scale  proved  a  somewhat  better  indicator  than  the  TEAMCMAQ  scale.  One 
equation  to  predict  ATM  performance  was  developed: 

TASK1071  =  (-.90)  ABSDIF  +  2.61 

The  above  equation  has  a  multiple  R  of  .44  and  explains  19  percent  of  the  variance  (F  is 
significant  at  the  p  <  .05).  As  will  be  shown,  this  is  the  only  equation,  including  both  "logical" 
and  factor  scales,  built  around  the  TASK1071  measure.  The  equation  can  be  interpreted  to 
indicate  that  as  the  more  similar  a  crew’s  attitude  regarding  crew  fallibility  is,  the  better  their 
rating  on  Task  1071. 

Using  the  three  CREWFAL  weights  to  predict  ACE  scores  resulted  in  no  equations  being 
developed. 

When  regressed  with  the  simulator  performance  variables,  one  equation  developed: 

THRTMAX  =  7.39  PCONLY  -I-  (-17.15) 

The  equation  had  a  multiple  R  equal  to  .35,  and  explained  12%  of  the  variance  (F  is  significant 
at  the  p  <  .14  level).  Since  the  THRTMAX  variable  measures  the  longest  duration  of  any  one 
threat  encounter,  lower  THRTMAX  values  signify  better  performance.  In  the  above  regression 
equation,  a  positive  Beta  coefficient  for  the  PCONLY  weight  indicates  that  a  higher  PC  score 
on  the  CREWFAL  measure  predicts  worse  performance.  Perhaps,  if  the  PC’s  CREWFAL  score 
is  high,  he  believes  his  fellow  crewmember  to  be  error-prone,  thus  he  fails  to  distribute 
workload,  becomes  task  saturated,  and  performs  poorly.  As  will  be  seen  in  other  analyses,  a 
positive  attitude  predicting  worse  performance  is  not  usually  the  case. 

C.rsw  Attitude  Combinations  of  HLPCMAO  Scale  to  Predict  Behavior/Performance 

The  three  weightings  of  the  HLPCMAQ  scales  were  regressed  onto  the  ATM,  ACE,  and 
performance  measures.  The  results  of  the  ATM  regressions  can  be  seen  in  Table  8.4-1. 
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Table  8.4-1 

Results  of  Stepwise  Regression  (First  Step  Only): 
HLPC34AQ  (3  Weights)  as  Independent  Variables  and 
ATM  Task  Measures  as  Dependent  Variables* 

(n*20) 


Equation 


Multiple  R 


1. )  ATMALL  =  (-.53)  ABSDIF  + 

2. )  ATM  13  =  (-.61)  ABSDIF  + 

3. )  ATM  12  =  (-.61)  ABSDIF  + 

4. )  BIGRADE  «  (-1.28)  ABSDIF 

5. )  TASK1071  =  NE 


2.66  *56 
2.79  -58 
2.82  -60 
+  2.87  *76 


*  In  all  equations,  F  is  significant  at  the  p  <  .01  level. 


%  Variance 


32 

34 

35 
58 


The  results  of  Table  8.4-1  indicate  that  the  ABSDIF  weight  is  the  most  predictive  of  the  ATM 
measures.  The  negative  coefficients  were  expected;  the  more  similar  (lower  absolute  difference) 
crewmembers  score  on  the  HLPCMAQ  scale,  the  higher  they  score  on  rated  ATM  Tasks.  A  link 
therefore  exists  between  similarity  of  crewmember  attitudes  on  the  HLPCMAQ  scale  and  good 
performance  on  ATM  Tasks. 

Two  of  the  HLPCMAQ  weights  yielded  equations  when  regressed  onto  the  ACE  measures. 
Table  8.4-2  shows  these  results.  In  all  equations,  F  is  significant  at  the  p  <  .15  level. 


Table  8.4-2 

Results  of  Stepwise  Regression  (First  Step  Only) : 
HLPCMAQ  (3  Weights)  as  Independent  Variables  and 
ACE  Checklist  Measures  as  Dependent  Variables+ 

(n=20) 


Ecruation 

1. )  ACEALL  »  (-.54)  PCONLY  +  6.47 

2. )  TEAMACE  =  NE 

3. )  XMNITOR  =  (-1.23)  ABSDIF  +  3.91 

4. )  INFOEXC  =  (-.84)  PCONLY  +8.08 

5. )  WORKMNG  =  (-.74)  ABSDIF  +3.65 

6. )  GLOBAL  =  (-.72)  ABSDIF  +  3.65 

+  Levels  of  significant  are:  ***  p  <  .01,  **  p  <  .05,  and  *  p  <  .10. 


Multiple  R 

%  Variance 

.39 

15  * 

.57 

33  **★ 

.58 

33  *** 

.40 

16  * 

.34 

11 

The  above  equations  also  appear  to  indicate  that  the  more  similar  the  attitude  of  the  two 
crewmembers  regarding  HLPCMAQ,  the  better  the  aircrew  will  score  on  XMINTOR, 
WORKMNG,  and  GLOBAL.  However,  when  considering  the  HLPCMAQ  score  of  only  the  PC, 
a  more  positive  score  predicts  worse  performance  on  ACEALL  and  INFOEXC.  This  finding  is 
interpreted  to  mean  that  crews  perform  better  if  they  agree  on,  and  hold  similar  attitudes 
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*  rHc  MT  PCMAO  but  the  PC’s  HLPCMAQ  attitude  is  insufficient  to  predict  good  crew 
^rfo^’an^  conipts  are  congruent  with  Otose  of  effective  aircrew  coordination. 

Two  equations  developed  from  the  regression  equations  using  the  three  HLPCMAQ  weights  to 
predict  simulator  performance  variables.  Table  8.4-3  shows  the  results. 


Table  8.4-3 

Results  of  Stepwise  Regression  (First 
HLPCMAQ  (3  Weights)  es  Independent  Varxebl^s  and 
Simulator  Perforsiance  as  Dependent  Variables+ 

(n*20)  .  ,  ^ 

Multiple  R 

n^quat2.on 


1. )  NAVTIME  ■  ("2.19)  PCANDPI  +  55.88 

2 .  )  DEVIATE#  *  NE 

3 .  )  %OFFCOUR  =  NE 

4 .  )  WITHIN  =  NE 

5 .  )  THRT#  =  NE 

6 .  )  THRTIME  =  NE 

7 .  )  THRTMAX  =  NE 

8. )  MEANDUR  “  NE 

9. )  ILSRIGHT  =  (-13.91)  ABSDIF  +  89.72 


.40 


%  Variance 


16  * 


.42 


18  * 


+  Levels  of  significance  are:  ***  p  <  -01,  **  p  <  -05,  and  *  p  <  .10. 


The  equation  developed  around  the  NAVTIME  performance  measure  indicates  that  when  Ae 
comb^ation  of  the  PC’s  and  Pi’s  attitude  yields  a  higher  HLPCMAQ  attitude,  the  ^ 
it  is  that  their  NAVTIME  performance  will  be  better.  The  equation  built  around 
ilidic^es  uJa^as  a  crew  holds  similar  attitudes  on  the  HLPCMAQ  scale,  the  percent  of  I^  steps 
correcUy  executed  increases.  This  latter  finding  again  demonstrates  that  the  homogeneity  of  the 
crew’s  attitude  positively  affects  performance. 

^  Prpu,  Attitude  Cnmhinations  of  niVEGET  Scale  to  Predict  Behavior/Performance 

Although  the  results  of  the  analyses  utilizing  the  three  GIVEGET  weights  did  not  yield  as  m^y 
equation  as  the  three  HLPCMAQ  weights,  the  conclusions  are  equally  as  interesttng.  The 
results  of  the  regressions  of  the  ATM  scales  with  the  three  GIVEGET  weights  can  be  seen  in 
Table  8.5-1.  Note  that  F  is  significant  at  the  p  <  .15  level  in  all  equations. 
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Table  8*5-1 

Results  of  Stepwise  Regression  (First  Step  Only): 
GIVEGET  (3  Weights)  as  Independent  Variables  and 
ATM  Task  Measures  as  Dependent  Variables+ 

(n=20) 

Equation  Multj-ple  R 


%  Variance 


1. )  ATMALL  »  (-.27)  ABSDIF  +  2.56 

2. )  ATM  13  =  NE 

3. )  ATM"12  =  (-.30)  ABSDIF  +  2.69 

4. )  BIGRADE  =  (-.57)  ABSDIF  +  2.56 

5. )  TASK1071  *  NE 


33 

11 

34 

11 

39 

16  * 

+  Levels  of  significance  are:  ***  p  <  .01,  **  p  <  .05,  and  *  p  <  .10. 


These  equations  indicate  that  the  more  diverse  the  attitudes  of  the  PC  and  PI  regarding  the 
giving  and  getting  of  information,  the  lower  their  score  will  be  on  the  ATM  Tasks.  These  results 
are  consistent  with  HLPCMAQ  scale:  the  more  similar  a  crew,  the  better  their  ^rformance  is 
likely  to  be.  It  appears  that  when  using  the  "logical"  scales,  the  ABSDIF  weighting  is  the  most 
powerful  predictor  combination  for  ATM  Task  performance. 

Of  the  six  ACE  measures  analyzed,  only  one  produced  a  significant  equation  with  the  GIVEGET 
scale: 


XMNTTOR  =  (-.80)  ABSDIF  -I-  3.78 

The  multiple  R  of  this  equation  equals  .43,  with  18%  of  the  variance  explained  (F  is  significant 
at  the  p  <  .06  level).  This  equation,  signifying  that  the  more  similar  a  crew’s  attitude,  the  better 
their  performance,  is  congruent  with  many  of  the  previous  findings  in  this  Section. 

The  three  GIVEGET  weights  yielded  more  significant  equations  with  the  dependent  simulator 
performance  variables  than  with  any  of  the  other  "logical"  scales.  Table  8.5-2  shows  these 
equations.  In  all  equations,  F  is  significant  at  the  p  <  .15  level. 
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Xabl*  8.5-2 

Results  of  Stepwise  Regression  (First  Step  Only) : 
GIVEGET  (3  Weights)  as  Independent  Variables  and 
Simulator  Performance  as  Dependent  Variables+ 

(n*20 ) 


32  ** 


Eduation  Multiple^  %  Variance 

1. )  NAVTIME  =  (-3.98)  PCOKLY  +  52.96  .43  19  * 

2 .  )  DEVIATE#  =  NE 

3. )  %OFFCOUR  =  NE  r,  oc  ** 

4. )  WITHIN  =  .42  PCONLY  +  (-2.12)  .51  26  ** 

5. )  THRT#  =  (-1.92)  PCONLY  +  15.13  .38  14 

6 .  )  THRTIME  *  NE 

7 .  )  THRTMAX  =  NE 

8.  )  MEANDUR  -  NE  -.o  ** 

9. )  ILSRIGHT  =  (-15.41)  ABSDIF  +  92.86  .56  32 

+  Levels  of  significance  are:  ***  p  <  .01,  **  p  <  .05,  and  *  p  <  .10. 

In  three  of  the  four  equations  developed,  the  PCONLY  weight  is  the  most  influential  of  the  three 
combinations.  Again,  note  the  inverse  nature  of  many  of  the  performance  measures  (specifically, 
NAVTIME,  DEVIATE#,  %OFFCOlJR,  THRT#,  THRTIME,  THRTMAX,  and  MEANDUR), 
where  lower  values  denote  better  performance.  The  equations  built  around  NAVTIME  and 
THRT#,  along  with  WITHIN,  all  indicate  that  performance  is  improved  when  the  PC  has  a  good 
attitude  about  the  exchange  of  mission  information  (GIVEGET)  within  a  crew.  Clearly,  the 
attitude  of  the  PC  is  an  important  determinant  of  successful  information  exchange. 

The  equation  developed  around  ILSRIGHT  demonstrates  that  the  ABSDIF  weight  is  most 
predictive  of  this  measure.  The  GIVEGETTLSRIGHT  relationship  seems  to  be  differentiated 
from  the  other  GIVEGET-performance  relationships  in  this  respect. 

8.6  Crew  Attitude  Combinations  of  CMOALL  Scale  to  Predict  Behavior/Performance 

The  three  weights  of  the  CMQALL  scale  were  regressed  with  the  ATM,  ACE,  and  performance 
measures.  One  equation,  shown  below,  was  developed  around  the  BIGRADE  variable: 

BIGRADE  =  (-.93)  ABSDIF  -f  2.58 

The  multiple  R  was  equal  to  .41,  and  17  percent  of  the  variance  was  explained  (F  was 
significant  at  the  p  <  .07  level).  This  equation  shows  that  the  more  similar  a  crew’s  attitudes, 
the  better  their  BIGRADE. 

One  equation,  shown  below,  was  developed  for  the  XMNTTOR  scale: 

XMNTTOR  =  (-1.10)  ABSDIF  -I-  3.72 
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The  multiple  R  was  equal  to  .38,  and  14  percent  of  the  variance  was  explained  (F  vm 
significant^at  the  p  <  .10  level).  Again,  this  equation  shows  that  the  mote  sinular  a  crew,  the 

better  their  performance. 

The  equations  developed  using  the  three  weights  of  the  CMQALL  and  fte  performan^  measuTK 
ate  iSsonably  consisSit  with  results  obtained  with  other  scales.  Table  8.6-1  shows  these  results. 
In  all  equations,  F  is  significant  at  the  p  <  .15  level. 


Tabls  8.6-1 

Rasul-bB  of  Stapwisa  Ragra*tion  (First  Step  Only): 
CMQftLL  (3  Weights)  ms  Independent  Variables  and 
Simulator  Performance  as  Dependent  Variables 

t  o  n  \ 


(n*20) 


lation 


Multiple  R 


%  Variance 


1.  )  NAVTIME  *  NE 

2 .  )  DEVIATE#  =  NE 

3 .  )  %OFFCOUR  =  NE 

4. )  WITHIN  *  NE  35  12 

5. )  THRT#  =  (-2.54)  PCONLY  +  17.82 

6 .  )  THRTIME  =  NE 

7 .  )  THRTMAX  =  NE 

8. )  MEANDUR  *  NE  co  .36  13 

9. )  ILSRIGHT  =  (-16.39)  ABSDIF  +  89.63 

+  Levels  of  significance  are:  ***  p  <  .01,  **  p  <  .05,  and  *  p  <  .10. 

In  Table  8.6-1  equations  show  that  the  PCONLY  weight  of  the  CMQALL  scale  positi^ly 
affects  THRT#  performance.  The  better  the  PC’s  attitude,  the  fewer  the  threats  encountered.  The 
SgHT  variable  is  positively  affected  when  the  PC  and  PI  have  more  similar  scores  on 

CMQALL. 


Results  presented  in  this  Section  clearly  demonstrate  that  various  combinations  of 
attitude  as  measured  by  the  Army  CMAQ,  are  able  to  predict  performance  on  ATM  Tasla, 
ACE  Checklist,  and  performance  measures.  Of  note  was  that  the  ABSDIF  weight,  essentially 
a  coefficient  of  agreement  among  the  two  crewmembers,  appeared  to  be  the  most  robust  weight. 
These  findings  are  of  critical  importance  to  the  understanding  of  aircrew  dynamics. 
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Section  9.0 

Relationships  Among  the  Measures  Using  the  CMAO  "Factor"  Scales: 

Various  Crew  Combinations 


9.1  Introduction 

The  Army  CMAQ  "factor"  scales  comprise  four  attribute  scales:  COMMCOR,  SHARLEAD, 
STRESS,  and  CMAQ34.  As  previously  described  in  Paragraph  8.1,  ten  combinations  of  the 
CMAQ  "factor"  scales  were  weighted  and  correlated  with  ATM,  ACE,  and  performance 
variables.  Appendix  F  shows  the  bivariate  correlation  matrix.  As  was  the  case  for  the  CMAQ 
"logical"  scales,  the  PCONLY,  PCANDPI,  and  ABSDIF  weights  displayed  the  strongest 
relationships;  therefore,  only  those  combinations  were  include  in  these  analyses. 

Stepwise  regression  equations  were  calculated  using  the  ATM,  ACE,  and  performance  variables 
as  dependent  variables  with  the  PCONLY,  PCANDPI,  and  ABSDIF  weights  as  the  independent 
variables.  As  discussed  in  Paragraph  8.1,  the  probabilities  for  F-to-enter  and  F-to-remove  from 
any  regression  equation  were  relaxed  to  the  p  <  .15  and  p  <  .16  levels,  respectively.  This 
permitted  development  of  meaningful  stepwise  regression  equations  using  the  three  CMAQ 
weights  with  the  dependent  variables.  Also,  as  previously  stated,  only  the  variable  which  entered 
on  the  first  step  is  displayed  in  the  following  tables. 

The  organizational  chart  for  this  Section  is  at  Table  9.1-1. 
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Table  9.1-1 

Organizational  Chart  for  Section  9 : 
Army  CMAQ  ** Factor**  Scales 


Analysis/Equation 

I nt erpr e t  at ion 

Table(s) 

9 . 2  Crew  attitude 
combinations  of  COMMCOR 
Scale  to  predict 
behavior / performance 

ABSDIF  predicts  ATM 

Tasks;  ABSDIF  and 

PCONLY  predict  ACE 
measures;  each 
weighting  enters  a 
performance  prediction 
equation 

9.2-1,  9.2-2,  9.2-3 

9.3  Crew  attitude 
combinations  of 

SHARLEAD  Scale  to 
predict  attitude/ 
performance 

Does  not  predict  ATM 

Task  performance; 

PCONLY  predicts  several 
ACE  measures ;  PCONLY 
and  ABSDIF  predict 
several  performance 
measures 

9.3-1,  9.3-2 

9.4  Crew  attitude 
combinations  of  STRESS 
Scale  to  predict 
at t it ude / performance 

9.4-1 

9.5  Crew  attitude 
combinations  of  CMAQ34 
Scale  to  predict 
attitude/performance 

Does  not  predict  ATM  or 
ACE  performance;  PCONLY 
predicts  NA.VTIME  and 
THRT#;  ABSDIF  predicts 
ILSRIGHT 

9.5-1 

9.2  Crew  Attitude  Combinations  of  COMMCOR  Scale  to  Predict  Behavior/Performance 

The  three  weights  of  the  COMMCOR  scale  yielded  many  regression  equations  across  the  ATM, 
ACE,  and  performance  variables.  Four  of  the  five  ATM  measures  yielded  equations  with  the 
ABSDIF  weight  entering  the  stepwise  regression  equation  on  the  first  step.  These  equations  are 
at  Table  9.2-1.  Where  an  equation  was  developed,  F  is  significant  at  the  p  <  .15  level. 
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Table  9.2-1 

Results  of  Stepwise  Regression  (First  Step  Only) : 

COHMCOR  (3  Weights)  as  Independent  Variables  and 
ATM  Task  Measures  as  Dependent  Variables-)- 

(n*20) 

Rmiation  Multiple  R  %  Variance 


1. )  ATMALL  =  (-.51)  ABSDIF  +2.64 

2. )  ATM  13  -  (-.54)  ABSDIF  -)-  2.74 

3. )  ATM  12  =  (-.52)  ABSDIF  +  2.76 

4. )  BIGRADE  =  (-.88)  ABSDIF  +  2.64 

5. )  TASK1071  =  NE 


43 

18 

* 

41 

17 

* 

40 

16 

★ 

41 

17 

* 

+  Levels  of  significance  are:  ***  p  <  .01,  **  p  <  .05  and  *  p  <  .10. 


When  crewmembers  held  more  similar  attitudes  (ABSDIF)  about  communication  and 
coordination  (COMMCOR),  they  tended  to  receive  higher  ATM  ratings.  This  finding  is  similar 
to  those  presented  in  Tables  8.4-1  and  8.5-1  for  the  HLPCMAQ  and  GIVEGET  logical  scales. 
The  finding  is  also  in  consonance  with  Table  2.5-3  wherein  GIVEGET  and  HLPCMAQ  are 
postulated  to  be  equivalent  to  COMMCOR. 

Four  of  the  six  ACE  scales  are  predicted  via  two  of  the  three  weights  of  the  COMMCOR  scale. 
Table  9.2-2  shows  the  results  of  the  equations.  F  is  significant  at  the  p  <  .15  level  in  all 
equations  developed. 


Tabl*  9.2-2 

Results  of  Stepwise  Regression  (First  Step  Only): 
COMMCOR  (3  Weights)  as  Independent  Variables  and 
ACE  Scales  as  Dependent  Variables-)- 

(n=20) 

Equation 

1. )  ACEALL  =  (-.76)  ABSDIF  +  3.68 

2. )  TEAMACE  =  NE 

3. )  XMNITOR  =  (-1.34)  ABSDIF  -)■  3.94 

4. )  INFOEXC  =  NE 

5. )  WORKMNG  =  .92  PCONLY  -)•  (-2.19) 

6. )  GLOBAL  =  (-1.16)  ABSDIF  +  3.85 

+  Levels  of  significance  are;  ***  p  <  .01,  *»  p  <  .05,  and  *  p  < 


Multiole  R 

%  Variance 

.38 

14  * 

.49 

24  ** 

.41 

17  * 

.43 

18  * 

.10. 


The  equations  presented  in  Table  9.2-2  are  similar  to  those  presented  in  Table  8.4-2.  In  this 
case,  ACEALL,  XMNITOR,  and  GLOBAL  measures  are  best  predicted  by  the  ABSDIF  weight. 
In  all  of  these  equations,  the  Beta  weight  of  ABSDIF  is  negative,  meaning  that  as  the  difference 
among  crewmembers’  COMMCOR  attitude  decreases,  performance  improves.  It  is  particularly 
interesting  to  note  that  this  is  also  true  for  the  GLOBAL  measure;  i.e.,  the  overall  performance 
rating  can  be  improved  if  crewmembers  hold  similar  attitudes. 
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The  equation  developed  around  the  WORKMNG  scale  indicates  that  of  the  three  weights  used, 
the  PCONLY  weight  has  the  most  importance  for  that  ACE  subscale  rating.  While  the  result  is 
statistically  significant  at  only  the  p  <  .10  level,  it  appears  that  the  PC’s  good  attitude  about 
COMMCOR  can  have  a  positive  effect  on  the  crew’s  performance  and,  in  particular,  on 
establishing  and  maintaining  reasonable  workload  levels. 

Table  9.2-3  shows  the  equations  generated  when  the  COMMCOR  scale  is  entered  using  the  three 
weights  as  independent  variables  and  the  performance  variables  as  the  dq)endent  variables.  F 
is  significant  at  the  p  <  .15  level  in  all  equations  developed. 


Xabl*  9.2-3 

Results  of  Stepwise  Regression  (First  Step  Only) : 
COMMCOR  (3  Weights)  as  Independent  Variables  and 
Simulator  Perforsuince  as  Dependent  Variables-t- 

(n*20) 


Equation 

1. )  NAVTIME  =  (-8.08)  PCONLY  +  78.03 

2. )  DEVIATE#  =  (-1.09)  PCONLY  +7.96 

3. )  %OFFCOUR  =  (-17.74)  PCANDPI  +  238.31 

4. )  WITHIN  *  NE 

5. )  THRT#  *  (-2.64)  PCONLY  +19.79 

6.  )  THRTIME  =  NE 

7 .  )  THRTMAX  =  NE 

8 .  )  MEANDOR  =  NE 

9. )  ILSRIGHT  =  (-16.85)  ABSDIF  +  91.20 


MultiiDle  R 

%  Variance 

.62 

38  *** 

.44 

19  ** 

.55 

30  ** 

.36 

13 

.42 

18  * 

+  Levels  of  significance  are:  ***  p  <  .01,  •*  p  <  .05,  and  *  p  <  .10. 


As  shown  by  Table  9.2-3,  all  three  different  weights  enter  the  equations  and  have  varying  levels 
of  predictive  value  when  regressed  with  the  performance  variables.  All  the  Beta  weights  are  in 
the  expected  direction  in  relation  to  the  variable  being  predicted.  The  equations  built  around  the 
NAVTIME,  DEVIATE#,  and  THRT#  variables  indicate  that  the  attitude  of  the  PC  regarding 
COMMCOR  has  the  most  positive  effect  on  crew  performance.  Similarly,  the  equation  built 
around  %OFFCOUR  signifies  that  the  more  positive  both  crewmembers  scores  are  on 
COMMCOR,  the  lower  their  %OFFCOUR.  The  equation  built  around  the  ILSRIGHT 
performance  measure  indicates  that  the  more  similar  a  crew’s  attitudes  regarding  communication 
and  coordination,  the  better  their  performance  will  be  on  this  measure.  This  latter  finding  is 
consistent  with  the  findings  of  Tables  8.4-3  and  8.5-2,  in  which  ABSDIF  was  the  best  predictor 
of  ILSRIGHT  on  both  the  HLPCMAQ  and  GIVEGET  scales.  The  finding  also  provides 
additional  supporting  evidence  to  substantiate  the  linkages  depicted  in  Table  2.5-3. 

9.3  Crew  Attitude  Combinations  of  SHARLEAD  Scale  to  Predict  Attitude/Performance 

When  the  SHARLEAD  scale  was  entered  into  regression  equations  using  the  three  weights  as 
the  independent  variables  with  ATM,  ACE,  and  performance  variables  as  the  dependent 
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variables,  the  results  again  varied  with  the  type  of  independent  measure.  No  equations  could  be 
developed  for  the  ATM  scales.  Two  of  the  six  ACE  measures  yielded  equations.  The  ACE 
equations  are  at  Table  9.3-1.  In  all  equations,  F  is  significant  at  the  p  <  .15  level. 


Tabl*  9.3-1 

Resul'bs  of  Stopwi««  Ragrassion  (Firat  S'fcap  Only): 
SHARliEAD  (3  Waights)  as  Indapandant  Variablas  and 
ACE  Chacklist  Maasuras  as  Dapendant  Variablas+ 

(n=20) 


Equation 

1 .  )  ACEALL  =  NE 

2 .  )  TEAMACE  =  NE 

3. )  XMNITOR  *  (-.52)  PCONLY  +  6.24 

4. )  INFOEXC  =  (-.45)  PCONLY  +  5.70 

5 .  )  WORKMNG  =  NE 

6 .  )  GLOBAL  =  NE 


Multiple  R  %  Variance 


.35  12 

.39  15  * 


+  Levels  of  significance  are:  ***  p  <  .01,  ***  p  <  .05,  and  *  p  <  .10. 


The  equations  built  around  the  XMNITOR  and  INFOEXC  measures  indicate  that  there  is  a 
relationship  between  the  PC’s  SHARLEAD  attitude  and  crew  j^rformance.  In  these  two 
equations,  it  appears  that  the  more  positive  the  PCONL\  attitude  is,  the  lower  the  ratings  on 
XMNITOR  and  INFOEXC;  i.e. ,  a  positive  PCONLY  attitude  regarding  the  sharing  of  leadership 
may  have  a  negative  impact  on  performance.  This  result  has  low  statistical  significance,  in  itself 
a  favorable  condition  since  it  is  a  difficult  relationship  to  explain  and  it  is  inconsistent  with  the 
principles  of  aircrew  coordination.  However,  it  may  indicate  that  a  PC  with  a  low  score  on 
SHARLEAD  is  an  autocratic  or  authoritarian  leader,  a  trait  which  for  some  unknown  reason  may 
be  helpful  in  improving  XMNINTOR  and  INFOEXC. 

The  results  of  the  regression  equations  with  the  three  weights  of  the  SHARLEAD  scales  and  the 
performance  variables  resulted  in  the  equations  at  Table  9.3-2.  In  all  equations,  F  is  significant 
at  the  p  <  .15  level. 
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Table  9.3-2 

Results  of  Stepwise  Regression  (First  Step  Only): 
SHARIiEAD  (3  Weight*)  a*  Independent  Verifies  and 
Simulator  Perfoxnaance  as  Dependent  Variables+ 

/  -—on \ 


Kauation 

1.)  NAVTIME  =  (-2.03)  PCANDPI  +  53.67 
2 ! )  DEVIATE#  =  NE 

3 .  )  %OFFCOUR  =  NE 

4. )  WITHIN  =  NE 

5. )  THRT#  =  (-1.83)  PCONLY  +  14.74 

6 .  )  THRTIME  =  NE 

7. )  THRTMAX  =  NE 

8. )  MEANDUR  *  NE 

9. )  ILSRIGHT  =  (-10.00)  PCONLY  +  140.51 

+  Levels  of  significance  are;  ***  p  <  .01, 


Multiple  R  %  Variance 


.40 


16  * 


.40  16  * 

.43  18  * 

**  p  <  .05,  and  *  p  <  .10. 


The  equation  buUt  around  NAVTIME  indicates  that  the  "better"  the  combined  attitudes  of  the 
PC  and  PI  are,  the  better  NAVTIME  will  be;  i.e. ,  performance  improves. 

The  THRT#  equation  indicates  that,  as  has  been  seen  in  other  regression  equations  using 
perfonnance  measures  as  the  dependent  variables,  the  PC’s  positive  attitude  is  positively  relat^ 
to  good  performance.  This  does  not  mean  that  the  PC  is  not  performmg  as  the  leader  of  the 
crew;  but,  rather  how  the  VC  feels  about  these  concepts  apparently  mfluences  the  atmosphere 
and  performance  of  the  crew. 

The  equation  developed  around  the  ILSRIGHT  measure  indicates  that  the  PC  s  attitude  can  have 
a  negative  effect  on  the  crew’s  ILSRIGHT  performance,  thereby  contradicting  the  equation  budt 
around  THRT#.  This  finding  is  also  different  from  all  previous  equations  develop^  around  the 
ILSRIGHT  measure.  A  plausible  explanation  may  be  that,  as  previously  determined  on  other 
attitude  scales,  the  primary  determinant  on  ILSRIGHT  performance  is  the  extent  to  w^h  the 
crewmembers  agree  (ABSDIF).  In  the  present  case,  a  negative  relationship  with  the  PCONLY 
weight  lends  credence  to  the  "crew  agreement"  finding,  i.e,  Aat  the  crew’s  combined  agreement 
on  an  attitude  is  most  important;  the  PC’s  attitude  alone  is  insufficient. 

In  summary,  the  SHARLEAD  factor  provided  some  "other  than  expected"  results.  One 
explanajrtion  may  be  that  for  Army  personnel  the  idea  of  "sharing  responsbility  for  leadership 
is  anathema  to  their  perceptions  of,  and  training  for,  leadership  responsibility.  This  idea 
regarding  the  PC’s  leadership  responsibility  may  come,  in  part,  from  AR  95-1  which  dictates 
that  the  PC  has  absolute  authority  in  the  cocl^it.  Aviators  who  reject  the  notion  of  sh^ed 
leadership  may  be  successful  in  terms  of  at  least  some  of  the  measures  available  in  this  study. 
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2^  r-„„.  A.rin,ri^  of  STRF.ss  Scalt  10  Pr'-'lin  ftttit'i'it/Ftrfopna'g? 

Th^  STRESS  CMAQ  scales  were  weighted  in  the  three  combinations  md  regre^ 

I™  ^E  ^d  irformance  measures.  No  equations  were  derived  for  tite  ATM  or  ACE 
m™ure1“  e  eqmtions  for  the  performance  variables  are  at  Table  9.4-1 .  F  ts  stgmficant  at  the 
p  <  .15  level  in  all  equations. 


Tabl#  9.4-1 

Results  of  Stepwise  Regression  (First  Step  Only): 
STRESS  (3  Weights)  as 

Simulator  Performance  as  Dependent  Variable 

(n“20)  ,  ,  ^ 

Multiple  R 

Equation 


1. )  NAVTIME  =  2.18  ABSDIF  +  28.11 
2 '.  )  DEVIATE#  =  NE 

3. )  %OFFCOUR  *  NE 

4.  WITHIN  -  (-.26)  ABSDIF  +  -52 

5. )  THRT#  =1.55  ABSDIF  +2.65 

6. )  THRTIME  =  22.33  ABSDIF  +31.91 

7. )  THRTMAX  =  6.55  ABSDIF  +  13.96 
8 !  )  MEANDUR  =  NE 

9 . )  ILSRIGHT  =  NE 


.34 


.46 

.45 

.44 

.41 


*  Variance 
12 


21  ** 
20  ** 
19  ** 
17  ** 


+  Levels  of  significance  are:  ***  p  <  -Olr  **  p  <  -05,  and  p 


<  .10. 


Table  9  4-1  demonstrates  that  ABSDIF  is  the  best  weighting  to  predict  the  i^rformance 
when  using  the  STRESS  scale.  When  crewmembers  hold  *<1 

recognition  of  stressor  effects,  the  performance  of  the  crew  improves  (NA^E,^T#,  md 
TTTRTMAX  are  inverse  variables,  with  good  performance  noted  by  lower  scores).  These  results 
^^XtTIttitude  of  recogking,  and  presumably 

reauires  a  different  type  of  aircrew  working  relationship  than  either  COMCO^ 
SHARLEAD.  Note  that  prediction  of  simulator  performance  usmg  abs  In 

8  resulted  in  similar  crew  attitude  weights  appearing  across  four  of  the  five 
the  case  of  the  "factor"  scales,  the  "best"  weights  are  different 

used  in  predicting  performance.  This  finding  points  to  the  conclusion  that  the  factor  scales  are 

most  likely  assessing  different  attitudes. 

2^  rrpw  Attitude  Combinations  of  rMA034  Scale  to  Predict  Attitude/Performance 
Using  the  three  different  weights,  the  CMAQ34  scale  was 

performance  measures.  No  equations  were  developed  for  either  a™  or  ^CE  m^u  ^T 

9  5-1  shows  the  results  of  stepwise  regression  with  the  three  weights  of  ^  Q 
performance  variables.  F  is  significant  at  the  p  <  .15  level  in  all  equations. 
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Xabl*  9.5-1 

Resul'ts  of  Stepwis*  Regression  (First  Step  Only): 
CMAQ34  (3  Weights)  as  Independent  Variables  and 
Simulator  Performance  as  Dependent  Variablos+ 

(n»20) 


Ecruation 

Multiple  R 

%  Variance 

NAVTIME  = 

(-4.35)  PCX)NLY  +  54.32 

.39 

15  * 

DEVIATE# 

=  NE 

%OFFCOUR 

=  NE 

WITHIN  = 

NE 

.39 

15  * 

THRT#  =  ( 

-2.43)  PCONLY  +  17.66 

THRTIME  = 

NE 

THRTMAX  = 

NE 

MEANDUR  * 

NE 

.38 

15 

ILSRIGHT 

=  (-14.97)  ABSDIF  +  90.05 

+  Levels  of  significance 


As  was  seen  in  the  COMMCOR  and  SHARLEAD  analyses,  Table  9.5-1  demonstrates  that 
PCONLY  appears  to  be  the  best  predictor  of  NAVTIME  and  the  THRT#.  The  negative 
coefficients  indicate  that  the  PC’s  positive  attitude  serves  to  improve  crew  performance  on  these 
measures;  i.e.,  the  "better"  the  PC’s  attitude,  the  better  wHl  be  the  crew’s  performance  as 
determined  by  NAVTIME  and  THRT#. 

The  equation  built  around  ILSRIGHT,  while  only  marginally  statisticaUy  significant,  is  consistent 
with  several  other  equations  developed  around  ILSRIGHT.  It  shows  that  the  more  similar  the 
attitudes  of  the  crewmembers,  the  better  their  performance  will  be  on  an  ILS  approach. 

9.6  Summary 

The  CMAQ  "factor"  scales,  in  the  various  weights,  can  predict  ATM,  ACE  and  performance 
measures.  Comparing  these  results  with  those  found  in  Section  8,  it  appears  that  using  the 
"factor"  scales  results  in  a  greater  number  of,  and  more  significant  predictions  of,  behavior  or 
performance.  It  was  also  noted  that  the  best  weighting  combination  of  the  "factor"  scales  varied 
depending  on  which  scale  was  used  to  predict  performance.  This  finding  points  to  the  conclusion 
that  the  three  "factor"  scales  are  assessing  different  attitudes;  perhaps  more  so  than  the  "logical" 
scales  are  able  to  do. 

All  three  crew  attitude  weights  provide  insight  into  the  manner  in  which  attitude  affects 
performance.  The  weight  that  appeared  to  be  the  most  powerful  predictor  was  ABSDIF  since 
it  appeared  in  the  most  equations.  ABSDIF  also  consistently  predicted  behavior  or  performance 
in  a  "correct,"  easily  explained  manner.  On  the  other  hand,  PCONLY  and  PCANDPI  yield^ 
inconsistent  and  often  difficult  to  explain  results.  The  meaning  of  this  finding  is  that  it  is 
important  that  crewmembers  agree.  It  does  not  mean  that  crewmembers  need  to  have  a  good 
attitude  as  measured  by  the  CMAQ  or  its  subscales;  simply  that  they  must  have  similar  views. 
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In  summary 
organization 


it  appears  that  the  "best"  organization  of  the  Army  CMAQ  is  the  "factor 
;  and  the  "best"  weighting  of  crew  attitudes  is  the  ABSDIF. 


tl 
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<;prtinn  10.0  -  Summary  and  Conclusions 


10.1  General 

The  previous  Sections  of  this  report  presented  the  findings  resulting  from  the  analysis  of 
altitudinal  behavioral,  and  performance  data  coUected  during  the  Spnng  md  Summer  of  1990. 
xAtanv  findings  were  discussed  and  several  hypotheses  previously  thought  to  be  true  logic^y 
^’^etlSyFOv"  so  Beginning  ^  Paragraph  10.2,  a  compUation  of  the  acttons 
taken  and  findings  made,  coUected  by  Section  number  is  presented.  FoUoi^g  the  surammes 
Serai  patterns  discerned  during  the  analyses  are  presented,  and  recommendations  are  made  for 

continued  research. 

10.2  .Sftcrion  2:  Properties  of  the  Army  CMAQ 

.  Re^evaluated  the  structure  of  the  Army  CMAQ  'logi^-  ‘°SSp“™SA^t 

•Get  Information-  were  combined  into  one  scale  similar  m  fo^t  to  the  Provid^A^t 
Help"  scale  previously  developed.  Redefined  'Values  Crew  as  Values  Team  o  . 

*  Uniquely  placed  all  Army  CMAQ  items  into  subscales. 

*  Computed  reliabiUfies  for  the  Army  CMAQ  "logical"  scales  using  Cronb^h’s  Mpha 
frange  from  .51  to  .78),  spUt-half  (range  from  .33  to  .66),  and  test-retest  (range  fro 
^40  to  .81)  algorithms.  Reliabilities  fell  in  the  "acceptable"  range. 

*  Performed  factor  analysis  on  the  CMAQ.  Developed  factor  sc^esba^  on  tore  deM 
criteria.  Renamed  Factor  2  to  be  "Shared  Leadership."  Developed  linkage  chart  fo  th 

"logical"  and  "factor"  scales. 

*  Computed  reliabilities  for  the  Army  CMAQ  "factor"  scales  using  Cronbach’s  Alpha  for 
the  80  aviators  (range  from  .67  to  .81),  Cronbach’s  Alpha  for  the  40  aviators  (i^ge 
from  .69  to  .85),  and  split-half  (range  from  .49  to  .68)  algorithms.  Better  reliabilities 
were  obtained  for  the  CMAQ  "factor"  scales  than  for  the  logical  scales. 

*  Compared  selected  CMAQ  responses  of  high  and  low  quality  (as  ^termmed  by 

quality  ratings)  and  high  and  low  performing  (as  determmed  * 

ATN^L  sLle)  Army  aviators  to  the  ratings,  as  document^  by  Helmreich  et  ^ 
(1986),  of  "superior"  commercial  aviators.  Comparison  revealed  Army  aviators  hold 
somewhat  similar  CMAQ  attitudes;  however  — 

o  Army  aviators  differ  from  commercial  aviators  on  certain  items;  e.g.,  "My 
decision  making  ability  is  as  good  in  emergencies  as  in  routine  imssion 
Sons."  Ar^y  av4rs  tend  to  agree  with  this  statement;  "superior 
commercial  aviators  do  not.  Agreement  with  this  item  contradicts  the  principles 
of  good  aircrew  coordination. 
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0  Several  items  showed  significant  differences  in  the  responses  between  high  and 
low  "quality"  Army  aviators,  and  between  high  and  low  "performing"  Army 
aviators.  Generally,  the  direction  of  the  difference  was  in  the  expected  direction, 
i.e.,  better  "quality"  or  "performing"  aviators  tended  to  agree  more  with  a 
principle  of  aircrew  coordination  than  the  lesser  "quality"  or  "performing"  Army 
aviators. 

10.3  Section  3:  Properties  of  the  ACE  Checklist 

*  Uniquely  placed  ACE  items  into  subscales. 

»  Computed  reliabilities  for  five  ACE  subscales.  Cronbach’s  Alphas  for  the  subscales 
ranged  from  .66  to  .90;  Cronbach’s  Alpha  for  the  ACE  is  .93.  Reliability  coefficients 
for  the  ACE  are  high. 

*  Performed  factor  analysis  on  the  ACE  data.  ACE  "factor  analytic"  scales  appeared  to 
measure  similar  attributes  as  the  ACE  "logic-based"  scales.  The  three  scales  of  the  factor 
analytic  model  were  described  as  "communication  and  group  climate,"  "workload  and 
performance  management,"  and  "cross  monitoring  by  crewmembers." 

*  Rejected  the  ACE  "factor  analytic"  scales  from  incorporation  into  further  ACE  analyses 
for  reasons  detailed  in  Paragraph  3.4. 

10.4  Section  4:  Properties  of  the  Revised  ATM  Tasks 

*  Proved,  through  reliability  analysis,  that  the  ATM  scales  are  highly  reliable.  Since 
missing  data  are  a  problem  with  the  data  base,  both  Cronbach’s  Alpha  and  the  Spearman- 
Brown  prophecy  formula  were  used  to  compute  reliability  coefficients.  ATM  Task 
subscale  reliabilities  ranged  from  .85  to  .90. 

*  Discovered,  through  analysis  of  the  use  of  Task  1071  Standards,  that  IPs  tended  to  use 
certain  Task  1071  Standards  more  than  others.  In  particular.  Standards  6  and  8  were  used 
most  often,  while  standards  4,  3,  2,  1,  and  9  were  also  frequently  used.  Referencing 
Task  1071  within  other  ATM  Task  Standards  was  found  to  be  efficient  and  informative. 

*  Found  that  Standard  10  was  not  utilized  because  aircrews  did  not  avail  themselves  of  the 
opportunity  to  accomplish  their  own  post-flight  debriefing,  there  was  little  time  during 
the  simulator  session  to  critique,  and  because  post-flight  debriefings  may  not  be  a  part 
of  the  "culture"  among  unit  aviators  participating  in  the  testbed.  Nevertheless,  IP-raters 
thought  Standard  10  to  be  an  important  aircrew  coordination-related  activity  and  should 
be  kept  as  a  Standard. 


86 


*  Recommended  the  word  "conflict"  not  be  used  in  future  Standards.  Instead,  the  phrase 
"difference  of  opinion"  should  be  used.  "Conflict"  was  originally  used  m  a 
psychological  manner  to  represent  a  "difference  of  opinion."  To  Army  personnel,  the 
word  "conflict"  is  interpreted  to  mean  a  physical  fight  or  military  action. 

10.5  gprtinn  5:  Properties  of  the  Performance  Measures 

*  Presented  only  an  overview  of  the  performance  variables  in  this  Section.  DRC  assumed 
that  Anacapa  provided  ARIARDA  with  a  report  discussing  the  construction  and 
properties  of  the  performance  variables. 

*  Presented  analyses  in  subsequent  Sections  showing  that  the  performance  variables  were 
effective  in  discriminating  between  high  and  low  performing  aircrews.  While  no  analyses 
of  the  internal  properties  of  the  performance  variables  were  undertaken,  this  findmg 
substantiates  the  objectivity  of  the  performance  variables  and  an  underlying  high 
reliability. 

10.6  <;pr.tion  6:  Relationships  Among  the  Measures 

NOTE:  Sections  6  through  9  focus  on  various  regression  equations  computed  to 
determine  the  relationships  among  the  measures  used  in  this  study.  Sections  6  and 
7  incorporate  40  CMAQ  "obser/ations"  (or  scores);  Sections  8  and  9  mcorporate 
20  CMAQ  observations  representing  a  combination  of  an  aircrew’s  CMAQ  score. 

Section  6  shows  analyses  performed  to  demonstrate  the  relationships  among  the  ATM,  ACE, 

performance  measures,  and  CMAQ  "logical"  scales.  Findings  were. 

*  ACE  subscales  are  highly  predictive  of  ATM  performance  (explaimng  as  much  as  66% 
of  the  variance). 

*  ACE  subscales  are  moderately  predictive  of  the  performance  variables  (explaining  as 
much  as  46%  of  the  variance  in  the  navigation-related  performance  variables). 

*  CMAQ  "logical"  scales  are  not  predictive  of  ATM  performance,  but  appear  to  have  a 
small  effect  on  aircrew  coordination-related  ATM  tasks. 

*  CMAQ  "logical"  scales  are  not  predictive  of  ACE  performance. 

*  CMAQ  "logical"  scales  are  not  predictive  of  the  performance  variables. 

*  The  combination  of  CMAQ  and  ACE  subscales  has  very  good  predictive  power  of  the 

ATM  scales  (explaining  as  much  as  71  %  of  the  variance  in  ATM  Task  performance). 
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*  The  combination  of  CMAQ  subscales,  ATMALL,  and  ACE  subscales  is  a  very  good 
predictor  of  performance  measures  (explaining  as  much  as  60%  of  the  variance  in  the 
navigation-related  performance  variables). 

*  When  the  CMAQ  "logical"  scales,  ATMALL,  and  the  ACE  subscales  are  entered  into 
stepwise  regression  equations,  at  least  one  subscale  from  each  measure  showed  itself  to 
be  predictive  of  a  performance  measure.  Of  particular  note  is  the  fact  that  an  attitude 
subscale  entered  a  stepwise  regression  indicating  its  significance. 

10.7  Section  7:  Relationships  Amone  the  Measures:  CMAQ  "Factor"  Scales 

Section  7  focused  only  on  the  relationships  of  the  (n=40)  CMAQ  "factor"  scales  with  the  ATM 

Tasks,  ACE  Checklist,  and  performance  variables.  Findings  were: 

*  CMAQ  "factor"  scales  have  insignificant  predictive  value  in  determining  ATM 
performance. 

*  CMAQ  "factor"  scales  are  not  predictive  of  ACE  performance. 

*  When  the  CMAQ  "factor"  scales  and  ACE  subscales  are  both  regressed  with  the  ATM 
scales,  the  results  are  very  similar  to  those  found  when  the  CMAQ  "logical"  scales  are 
utilized. 

*  A  combination  of  CMAQ34  and  ACEALL  are  strong  predictors  of  ATM  Task 
performance  (explaining  as  much  as  67%  of  the  variance  in  the  ATM  scales).  These 
results  are  nearly  identical  to  those  using  CMQALL  and  ACEALL  to  predict  ATM  Task 
performance. 

*  The  CMAQ  "factor"  scales  do  have  predictive  value  when  regressed  with  the 
performance  variables,  demonstrating  an  empirical  link  between  attitudes  and 
performance.  Two  of  the  navigation -related  performance  variables  are  significantly 
predicted  (explaining  17%  and  20%  of  the  variance). 

*  When  the  combination  of  CMAQ  "factor"  scales,  ATMALL,  and  ACE  subscales  are 
regressed  with  the  performance  variables,  the  results  are  similar  to  the  results  utilizing 
the  "logical"  scales.  However,  the  use  of  the  CMAQ  "factor"  scales  versus  the  "logical" 
scales  results  in  a  greater  number  of  statistically  significant  equations  being  developed. 

*  Sections  6  and  7  demonstrate  that  a  measurable  link  exists  between  attitudes  (as 
determined  by  the  Army  CMAQ)  and  performance. 
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IQ  g  qprrinn  «:  T?p-1atinnships  Among  the  Measures  Using  the  CMAQ  "Logical"  Scales: 
v^riniis  Crew  Combinations 

Sections  6  and  7  proved  that  a  statistical,  measurable  relationship  exists  between  attitudes  and 
performance;  therefore,  Sections  8  and  9  investigated  the  hypothesis  that  some  combination  of 
a  crew’s  score  with  respect  to  attitudes  would  be  a  better  predictor  of  performance.  Ten 
combinations  of  crew  scores  were  developed  and  correlated  with  ATM,  ACE,  and  performance 
measures.  Three  of  the  combinations,  PCONLY,  PCANDPI,  and  ABSDIF  were  selected  for 
further  examination  based  on  their  correlations  wiA  the  other  measures.  In  Section  8,  the  CMAQ 
"logical"  scales  were  entered  into  the  combinations;  Section  9  used  the  CMAQ  "factor"  scales. 
Findings  were  that  when  crew  attitude  combinations  were  computed  for: 

*  TEAMCMAQ:  No  predictive  equations  were  developed. 

*  CREWFAL:  PCONLY  negatively  predicted  THRTMAX;  ABSDIF  predicted  TASK107 1 , 
the  only  measure  of  both  "logical"  and  "factor"  CMAQ  scales  to  predict  this  ATM  Task. 

*  HLPCMAQ:  ABSDIF  consistently  predicted  ATM  Task  measures;  PCONLY  or 
ABSDIF  predicted  several  ACE  measures;  PCANDPI  predicted  NAVTIME;  ABSDIF 
predicted  ILSRIGHT. 

*  GIVEGET:  ABSDIF  predicted  three  of  the  five  ATM  Task  measures  and  the  ACE 
subscale,  XMNTTOR;  either  PCONLY  or  ABSDIF  predicted  several  performance 
variables. 

»  CMQALL:  ABSDIF  predicted  BIGRADE,  XMNTTOR,  and  ILSRIGHT;  PCONLY 
predicted  THRT^. 

10.9  Section  9:  Relationships  Among  the  Measures  Using  the  CMAQ  "Factor"  Scales: 
Various  Crew  Combinations 

Section  9  focused  on  combinations  of  aircrew  attitudes  to  assess  the  relationship  between 
attitudes  and  behavior/performance.  In  Section  9,  the  CMAQ  "factor"  scales,  weighted  via 
combinations  of  aircrew  CMAQ  scores,  were  entered  into  the  regression  equations  used  in 
Section  8.  Findings  were  that  when  crew  attitude  combination  scores  were  computed  for: 

*  COMMCOR:  ABSDIF  best  predicted  ATM  Tasks;  ABSDIF  and  PCONLY  best 
predicted  the  ACE  measures;  each  of  the  three  weights  entered  into  an  equation  as  the 
best  predictor  of  a  performance  variable.  These  results  were  similar  to  those  obtained  for 
the  HLPCMAQ  and  GIVEGET  "logical"  scales. 

SHARLEAD:  No  predictive  equations  were  developed  for  ATM  Tasks;  PCONLY 
predicted  several  ACE  measures;  ABSDIF  and  PCONLY  predicted  many  of  the 
performance  measure  variables. 
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?!  XMNITOR  and  INFOEXC  measures 

indicate  ftat  there  is  a  relationship  between  the  PC’s  SHARLEAD  attitude  and 

PCOMLY°Immd“  ",?f  the  more  positive  the 

PCONLY  attitude  is,  the  lower  the  ratings  on  XMNITOR  and  INFOEXC-  i  f> 

a  po^tive  PCONLY  attitude  regarding  the  sharing  of  1  Jers^mafh^ie  a 

performance.  It  may  indicate  that  a  PC  with  a  low  score  on 
SHARLEAD  is  an  autocratic  or  authoritarian  leader,  traits  which  for  some 
unknown  reason  may  be  helpful  in  improving  XMNITOR  and  INFOEXC. 

m™  olX^Sel  Pt^hicted 

CMAQ34:  no  equations  were  develooed  for  atk/i  nr  Arr:  r..^ 

predicted  NAVTTME  and  TTOT#;  ABSOT  p^^™iSr?got”*' 

Several  disoemable  patterns  evolved  during  the  Section  9  analysis  of  the  -factor"  scales: 

All  three  crew  attitude  weights  (PCONLY  PCANnpi  j  • 

the  manner  m  Which  attitude  Xts  SS  Saf~Tbet 

performance  in  a"  -correct, 

inconsistent,  often  difficult  to  explain  results. 

and  GIVEGET,  perform  in  a  manner  similar 

va^te’NAVT^,  S^^"l,d  TOFFCO^^l'^"”^  For  example,  the 

for“the  ST“™Ix“"l,d 

THRTTME  which  are  all  threat-related  variahipc  tt  cpiru-r  «-a-iviaa,  and 

ru^sS?rw«£€- 

aircrew  attitude.  ^  ^ving  a  significant  relationship  to  at  least  one 

One  explanation  may  be 

to  their  pe^DW  of  °f /hanng  responstbility  for  leadership-  is  anathema 

the  nodOT  o/shared  ieart..r!r  aPPears  that  aviators  who  reject 

tarn  ^2iote’'le^„“’  *'i?Kn  ‘  ‘^'^‘334,  behave  somewhat  differendy 

i-e.,  each  of  fte  three^S!^^'*''^,*®  different  attributes! 

three  scales  seems  to  measure  unique  attitudinal  dimensions. 
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*  Conversely,  the  CMAQ  "logical"  scales  are  inconsistent  in  depicting  the  attitude  -*■ 
behavior/performance  relationship.  TEAMCMAQ  appears  to  be  the  weakest  subscale; 
CREWFAL  also  appears  to  be  relatively  weak.  The  GIVEGET  and  HLPCMAQ  scales 
are  more  robust,  but  the  results  of  these  analyses  are  similar  to  one  another  and  similar 
to  the  COMMCOR  "factor"  scale. 

10.10  Answers  to  the  Research  Questions 

Ten  research  questions  were  posed  in  Paragraph  6.1.  The  questions  and  their  answers  are 
provided  below.  Note  that  the  "variance  explained"  statements  following  each  answer  are  taken 
from  various  tables,  scales/subscales,  and  crew  attitude  combinations.  Where  shown,  for  the  % 
of  Variance  column,  ***  p  <  .01;  **  p  <  .05;  *  p  <  .10. 


Question  1.  What  is  the  relationship  between  the  two  measures  of  crew  behavior  (ACE 
Checklist  and  ATM  Tasks)? 

Answer  1.  The  two  behavior  measures  are  strongly  related  (as  much  as  66%  of  the  variance 
is  explained). 


Example  1.  From  Table  6.2-1 

ATM  13  «  .21  WORKMNG  +  .21  XMNITOR  + 

(-.19)  INFOEXC  +  .21  TEAMACE  +  .96 


Multiple  R  %  of  Variance 


.81  66% 


Question  2.  What  is  the  relationship  between  crew  coordination  behaviors  (ACE  Checklist) 
and  Mission  Performance? 

Answer  2.  Behavior  and  Mission  Performance  are  related  (as  much  as  46%  of  the  variance 
is  explained). 


Example  2.  From  Table  6.6-1 

NAVTIME  =  (-4.27)  WORKMNG  +1.97  XMNITOR  + 

.74  INFOEXC  +  (-1.72)  TEAMACE  +  41.55 


Multiple  R 


%  of  Variance 


.67 


46%  ** 


Question  3.  What  is  the  relationship  between  crew  behaviors  (ATM  Tasks)  and  Mission 
Performance? 

Answer  3.  Behavior  and  Mission  Performance  are  related  (as  much  as  27%  of  the  variance 
is  explained). 

Example  3.  From  Table  6.9-2 

Multiple  R  %  of  Variance 

DEVIATE#  =  (-1.15)  ATMALL  +  4.24  .52  27%  *** 


Question  4,  What  is  the  relationship  between  the  combined  effect  of  crew  coordination 
behaviors  (ACE  Checklist  +  ATM  Tasks)  and  Mission  Performance? 

Answer  4.  The  combined  effect  qf  the  crew  coordination  behaviors  is  highly  related  to 
Mission  Performance  (as  much  as  50%  of  the  variance  is  explained). 

Example  4.  New  equation,  no  reference  -  "forced  entry,"  (n=20) 

.  Multiple  R  %  of  Variance 

DEVIATE#  =  (-.40)  WORKMNG  +  (-1.30)  ATMALL  + 

(-.64)  INFOEXC  +  .65  XMNITOR  + 

.19  TEAMACE  +5.10  .71  50  * 


Question  5.  Which  organization  of  the  Army  CMAQ,  "logical"  or  "factor,"  is  better? 

Answer  5.  The  "factor"  organization  is  better.  Reliability  coefficients  are  higher  than  those 
of  the  "logical"  scales;  more  significant  relationships  (equations)  are  depicted 
when  using  the  "factor"  organization;  and  the  three  "factor"  scales  act  differently 
from  one  another  when  correlated  with  external  variables. 
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Question  6.  What  combination  of  crewmember  attitudes,  as  measured  by  the  Army  CMAQ, 
best  demonstrates  relationships  between  crew  attitude  and  crew  coordination 
behaviors/Mission  Performance? 

Answer  6.  The  absolute  difference  (ABSDIF) ,  which  is  essentially  a  coefficient  of  agreement 
between  the  two  crewmembers,  is  best.  As  the  crewmembers’  scores  on  an  Army 
CMAQ  attitude  dimension  become  more  similar,  crew  coordination  behavior  and 
Mission  Performance  tend  to  improve. 

NOTE:  To  more  precisely  answer  Questions  7-10,  new  equations 
were  computed  utilizing  the  three  Army  CMAQ  "factor"  scales 
(COMMCOR,  SHARLEAD  and  STRESS)  weighted  by  ABSDIF. 


Question  7.  What  is  the  relationship  between  attitudes  toward  crew  coordination  (Army 
CMAQ)  and  crew  coordination  behaviors  (ACE  Checklist)? 

Answer  7.  Attitude  and  behavior  are  related  (as  much  as  28%  of  the  variance  is  explained). 

Example  7.  New  equation,  no  reference  -  "forced  entry,"  (n=20),  ABSDIF  weight 

Multiple  R  %  of  Variance 

XMNITOR  =  .22  STRESS  +  (-1.28)  COMMCOR  + 

(-.16)  SHARLEAD  +  3.82  .53  28% 


Question  8.  What  is  the  relationship  between  crew  coordination  attitudes  (Army  CMAQ)  and 
crew  coordination  behaviors  (ATM  Tasks)? 

Answer  8.  Attitude  and  behavior  are  related  (as  much  as  20%  of  the  variance  is  explained). 
Example  8.  New  equation,  no  reference  -  "forced  entry,"  (n=20),  ABSDIF  weight 

Multiple  R  %  of  Variance 

ATM_13  =  (-.01)  STRESS  +  (-.51)  COMMCOR  + 

~  (-.12)  SHARLEAD  +  2.84  .44  20% 
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Question  9.  What  is  the  relationship  between  crew  coordination  attitudes  (Army  CMAQ)  and 
Mission  Performance? 


Answer  9.  Attitudes  and  Mission  Performance  are  related  (as  much  as  32%  of  the  variance 
is  explained). 


Example  9.  New  equation,  no  reference  -  "forced 

THRTMAX  =  7.01  STRESS  +  9.80  COMMCOR  + 
(-7.88)  SHARLEAD  +  14.28 


entry,"  (n=20),  ABSDIF  weight 

Multiple  R  %  of  Variance 
.56  32% 


Question  10.  What  is  the  relationship  between  the  combined  effect  of  crew  coordination 
attitudes  and  behaviors  (Army  CMAQ  +  ACE  Checklist  +  ATM  Tasks)  and 
Mission  Performance? 

Answer  10.  The  combined  effect  of  the  crew  coordination  measures  is  strongly  related  to 
mission  performance  (as  much  as  65%  of  the  variance  is  explained). 

Example  10a.  New  equation,  no  reference  -  "forced  entry,"  (n=20),  ABSDIF  weight 

Multiple  R  %  of  Variance 

NAVTIME  =  (-1.47)  ATMALL  -I-  2.59  STRESS  -I- 

1.49  SHARLEAD  +  (-5.67)  COMMCOR  -l- 
1.24  INFOEXC  -I-  (-4.94)  WORKMNG  -I- 

1.44  TEAMACE  -I-  (-.69)  XMNTTOR  -I-  42.39  .80  65% 


Example  10b.  New  equation,  no  reference  -  "stepwise  entry,"  (n=20),  ABSDIF  weight 

Multiple  R  %  of  Variance 


NAVTIME  =  (-4.14)  WORKMNG  +2.19  STRESS  + 
(-4.21)  COMMCOR  +  43.79 


.76  57%  +** 


10.11  Discussion  and  Recommendations 

*  The  CMAQ  "factor"  scales  should  be  used  in  future  studies.  Any  improvements  or 
revisions  made  to  the  Army  CMAQ  should  be  made  with  the  underlying  factor 
structure  in  mind. 

*  The  finding  that  crewmember  agreement  on  attitude  dimensions  is  a  predictor  of 
performance  needs  additional  investigation  to  more  fiilly  understand  this  relationship. 
This  concept  is  congruent  wdth  past  research  that  focuses  on  intracrew  "familiarity"  and 
"shared  mental  models."  Examples  of  this  research  can  be  found  in  Chidester,  et  al. 
(1990),  Kanki,  et  al.  (1989a  and  1989b),  Orasanu  (1990),  and  Thorsden,  et  al.  (1990). 
These  researchers,  however,  focused  primarily  on  operations-relevant  interactions.  The 
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importance  of  the  present  finding  is  that  shared  mental  models  may  weU  extend  to 
attitudes  and  perhaps  even  personality  as  well.  The  result  of  this  finding  may  be  that  if 
we  combine  crews  who  view  the  world  similarly  and  think  (and  behave)  in  ways  that  are 
expected  by  their  fellow  crewmembers,  performance  (and  ipso  facto,  safety)  is  enhanced. 

*  To  take  advantage  of  the  principle  that  "familiarity  and/or  agreement  breeds  good 
performance, "  the  Army  should  take  steps  to  indoctrinate  aviators  to  value  good  crew 
coordination.  Over  the  last  decade  there  has  been  much  research  to  substantiate  the 
concept  that  good  crew  coordination  improves  performance  (cf.  Povenmire  et  al  (1989) 
Helmreich  &  Foushee  (1988)  and  Chidester  et  al.  (1990)).  There  is  also  an  abundance 
of  evidence  gleaned  from  accident  investigations  showing  that  a  lack  of  effective  cockpit 
resource  management/aircrew  coordination  has  led  to  catastrophic  results. 

NOTE:  The  "agreement"  finding  discussed  in  the  two  preceding  points  could  be 
interpreted  to  mean  that  a  bad  attitude  is  tolerable  as  long  as  both  crewmembers 
share  it.  But  this  is  a  faulty  interpretation.  For  example.  Army  aviators  tend  to 
agree  with  the  statement,  "My  decision  making  abiUty  is  as  good  in  emergencies 
as  in  routine  mission  situations."  This  type  of  thinking  is  potentially  quite 
dangerous;  it  could  lead  to  a  false  sense  of  over-confidence  in  one’s  individual 
abilities  and  thus  lead  a  pilot  into  ignoring  (or  not  soliciting)  input  fi-om  his  crew 
dunng  critical  maneuvers  or  situations.  If  aviators  were  trained  in  the  principles 
of  aircrew  coordination,  they  would  1)  hold  attitudes  similar  to  one  another’s,  and 
2)  hold  attitudes  lending  themselves  to  aviation  safety. 

Given  that  a  close  relationship  exists  between  the  ACE  Checklist  and  ATM  Task 
m^ures;  i.e.,  they  are  both  behavior  ratings  and  they  are  both  highly  correlated  with 
mssion  performance,  the  Army  should  consider  integrating  the  ACE  Checklist  into  the 
APART  program.  The  findings  in  this  report  support  DRC’s  contention  that  both 

Ilia’S?  A™  Tasks  measure  fine-grained,  task-oriented  behavior; 

the  ACE  Checkhst  measures  ^  aircrew’s  ability  to  integrate  a  variety  of  human  factors 
prmciples  mto  the  cockpit  milieu.  A  two-perspective  evaluation  scheme  utilizing  the  two 
measures  would  capture  a  more  reaUstic  spectrum  of  aviator/crew  performance. 

NOra:  As  a  result  of  the  efficacy  of  the  ACE  Checkhst  demonstrated  by  the 
DRC/ARIARDA  work,  American  Airlines  (Treadway  Sc  Chidester,  1991, 
personal  communication)  is  considering  integrating  a  task  similar  to  Task  1071 
mto  their  maneuver/procedures  (the  commercial  corollary  to  ATM  Tasks)  based 
on  the  ACE  Checkhst  format.  This  approach  exemplifies  the  recommendation 


Imphcations  for  instruction: 

0  Organizing  instruction  around  the  concepts  embodied  in  the  RICS  Model  proposed 
development  pf  Measures  technical  report  appears  warranted. 


95 


The  CMAQ  "factor 
attitudes. 

The  ACE  dimensions 
performance. 


"  dimensions  should  be  used  as  organizers  for  teaching 


Incorporatmg  aircrew 
are  taught  is  highly  recommended 


should  be  useful  instructional  concepts.  They  are  related  to 
coordination  considerations  into  the  way  that  ATM  Tasks 


ImpUcations  for  flight  training  candidate  and  crew  selection; 


A  •  4.-  /tt^aaVNO  conducts  an  Initial  Entry  Rotary 

instruction  relying  on  a  Multi-TraA  W  « 
Using  the  lERW-MT  concept,  student  Vision  is 

^lybS«r>^^Ars^n^placement'batte.y.O^^^ 

O  consideration  should  be  given  to  the  ^^y^nt  h" 

fl.e  purpose  of  Mng  as  it  relates  to  certain  key 

acceptability  would  be  b^ed  ®  CMAQ-type  instrument  to  all  umt 

attitudes  found  as  a  result  of  admimstenng  a 

aviators. 

FoUow^n  studies  to  this  report  should  have  a  larger  sample  sire. 

.  fho  A  rp  rhprklist  This  factor  structure  should 
There  is  an  underlying  factor  structure  to  the  A  ■  testbed 

be  determined  on  a  larger  sample  than  was  available  for  the  testbed. 

Inter-rater  reliability  should  be  deten^ed  to  vldtoB™ 

Ift  «bS t“s:' A“;robJem 

good. 
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A  foUow-on  to  this  study  (or  ^  exploratoiy  study  using  the  cuiient  data)  could  profitably 
invesugate  the  relauonship  of  the  ACE  Checklist,  ATM  Tasks,  and  Army  CMAO  tdft 
three  cmegones  (or,  "macro"-variables)  of  mission  performance:  naviL^rda^ 
to^t-reiated  and  mstrument  flight-related.  It  is  thought  that  navigation  Jtd  instruS 
flight  are  especially  aircrew  coordination-intensive. 

Relationships  of  attitude  -  behavior  -  performance  using  all  three  CMAO  "factor"  scales 
weighty  with  the  ABSDIF  combination  could  be  investigated  more  thorougUy 

xiditional^tatistitS 

^e  a^tijde  -*  beha^vior  -  performance  linkage  should  be  investigated  further  It  is 
thought  that  the  linkage  may  change  based  on  the  problem  conL^orTStionS 
environment.  For  example,  were  the  mission  in  an  actual  instead  of  a  SiS 
environment,  would  performance  and  its  link  to  the  other  measures  have  been  different 

Attitudes  and  skills  change  over  time  and  are  greatly  influenced  by  an  individual’s 

measured.  Several  questions  could  then  be  answered. 

o  What  is  the  interaction  of  attitudes  and  skiUs? 

“  reftwher  training  in  aircrew  coordination  skiUs  take 

o  Does  combat  experience  affect  attitudes  or  skills  related  to  aircrew  coordination? 
^^worthwhile  research  questions  can  be  asked  regarding  the  Army  CMAQ.  For 


0 

o 


^  influence  CMAQ  responses? 

ould  other  me^ures  of  attitudes  or  personality  serve  as  better  predictors  of 
aircrew  coordination-related  behaviors? 

CampbeU  testbed  represents  the  initial  use  of  the  measures  and 

a  larTef  more^efinL^  procedures  in  preparation  for 

Armv  ic’  re^ed  testbed  mcorporating  improved  instruments  and  procSiures  The 
^y  IS  now  well;positioned  to  conduct  such  a  foUow-on  testbed  -  a  S  wWch  if 

attimde^  b^h^^  mdication,  should  produce  enhanced  empirical  definitions  of  the 
attitude  behavior  -*  performance  relationships  introduced  during  this  study. 
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APPENDIX  A 


Aircrew  Coordination  Measures 


A-1 


A-2 


i 

i~ 


APPENDIX  A.l 

Army  CMAQ 


A.  1-1 


A.  1-2 


Army  Aviation  Crewmember  Questionnaire 


Rev.  4 


Background  Infonnation 

(Please  complete  the  following  information  regarding  your  personal  experiences  and  current  status.) 


1.  Aviation  Experience  (Ht.  Hrs.) 


2.  Date  (daylmolyr) 


Lifetime  Hying  Experience 

All  NV  Devices 

Conditions  (e.g.,  NVG) 


Experience  over  last  6  months 

All  NV  Devices 

Conditions  (e.g.  NVG) 


a.  UH-60  hrs. 

b.  R/W  hrs. 

c.  Fixed  Wing  hrs. 


3.  Current  Rank  _ 

4.  Current  Unit  (Co/Bn/Rgt)  _ 

5.  Time  in  Current  Unit  (months) 

6.  Current  Aviator  Readiness  Level  (RL)  12  3  (circle  one  number) 

7.  Current  primary  duty  assignment  in  unit  (check  one): 

IP _  SP _  UT _  IFE _  MTP _  Aviator _  Other _ 


8.  Are  you  flight  lead  qualified  (circle  one):  Yes  No 

9.  Have  you  had  Aircrew  Coordination  Training?  Y  or  N  (circle  one:  if  yes,  answer  below .) 

Describe  ACT  training  experiences:  Course  title,  location  of  training,  approximate  date,  #  of 
hours  of  instruction,  quality  of  course. 

a.  Experience  #1; _ _ _ _ _ 


b.  Experience  #2: 


10.  Cross-Indexing  Code  (Note:  Your  responses  to  this  form  will  not  be  used  to  evaluate  you  and  will 

not  become  a  part  of  any  permanent  record  relating  to  you.  An  individual  identifier  is 
necessary  since  you  will  be  undertaking  other  related  activities  and  we  simply  need  a 
"cross-index"  number  to  keep  track  of  the  participants  in  this  research.) 


Social  Security  #: 


A. 1-3 


229 


Rev.  4 


IL  Opinion  Survey 

(Please  circle  the  number  on  the  agree-disagree  dimension  that  best  reflects  your  personaJ  attitude  toward  each  statement. 


There  are  no  ’’right"  or  "wrong"  answers.  We  are  simply  asking  for  your  honest  opinions.) 

Strongly  Slightly 

Disagree  Disagree  Disagree  Neutral 

Slightly 

Agree 

Agree 

strongly 

Agree 

1.  Crewmembers  should  avoid  disagreeing  with 
others  because  conflicts  create  tension  and  reduce 
crew  effectiveness. 

1 

2 

3 

4 

5 

6 

7 

2.  Crewmembers  should  feel  obligated  to  mention  their 
own  psychological  stress  or  physical  problems  to 
other  crewmembers  before  or  during  a  mission. 

1 

2 

3 

4 

5 

6 

7 

3.  It  is  important  to  comment  about  the  procedures  and 
techniques  of  other  crewmembers. 

1 

2 

3 

4 

5 

6 

7 

4.  Pilots-in-command  should  not  dictate  flight 
techniques  to  other  crewmembers. 

1 

2 

3 

4 

5 

6 

7 

5.  Casual  social  conversation  during  periods  of  low 
workload  can  improve  crew  coordination. 

1 

2 

3 

4 

5 

6 

7 

6.  Each  crewmember  should  monitor  other  crew¬ 
members  for  signs  of  stress  or  fatigue^  and  should 
discuss  the  situation  with  the  crewmember. 

1 

2 

3 

4 

5 

6 

7 

7.  Good  communications  and  crew  coordination  are  as 
important  as  technical  proficiency  for  the  safety  of 
the  flight. 

1 

2 

3 

4 

5 

6 

7 

8.  Crewmembers  should  be  aware  of  and  sensitive  to 
the  personal  problems  of  other  crewmembers. 

1 

2 

3 

4 

5 

6 

7 

9.  The  pilot-in-command,  time  and  situation  permitting, 
should  take  control  and  fly  the  aircraft  in  all 
emergency  and  non-standard  situations. 

1 

2 

3 

4 

5 

6 

7 

10.  The  pilot  flying  the  aircraft  should  verbalize  plans 
for  procedures  or  maneuvers  and  should  be  sure  that 
the  information  is  understood  and  acknowledged  by 
crewmembers  affected. 

1 

2 

3 

4 

5 

6 

7 

11.  Pilots  and  other  crewmembers  should  not  question 
the  decisions  or  actions  of  the  pilot-in-command 
except  when  these  actions  obviously  threaten  the 
safety  of  the  flight. 

1 

2 

3 

4 

5 

6 

7 

12.  Even  when  fatigued,  I  perform  effectively  during 
most  critical  flight  maneuvers. 

1 

2 

3 

4 

5 

6 

7 

13.  Pilots-in-command  should  encourage  pilots  and  crew 
chiefs  to  question  procedures  and  flight  profile 
deviations  during  normal  flight  operations  and  in 
emergencies. 

1 

2 

3 

4 

5 

6 

7 

14.  There  are  no  circumstances  where  the  pilot  should 

1 

3 

4 

6 

7 

take  the  aircraft  controls  without  being  directed  to  do 
so  by  the  pilot-in-command. 


A.  1-4 


229 

I 

I 


15. 


I 


16. 


17. 


18. 


19. 


20. 


21. 


22. 


23. 


24. 


25. 


26. 


27. 


28. 


29. 


30. 


Strongly  Slightly  Slightly 

Disagree  Disagree  DU^ee  Neutral  Agree 


A  debriefing  and  critique  of  procedures  and  1 

decisions  after  each  mission  is  an  important  part  of 
developing  and  maintaining  effective  crew 
coordination. 

Training  is  one  of  the  pilot-in-command's  1 

important  responsibilities. 


Under  high  stress,  good  crew  coordination  is  more  1 

important  than  it  is  under  low  stress  conditions. 

Effective  crew  coordination  requires  crewmembers  to  1 
take  into  account  the  personalities  of  other 
crewmembers. 

The  pilot-in-command*s  responsibilities  include  1 

coordination  of  inflight  crew  chief  activities. 

Most  crewmembers  can  leave  personal  problems  1 

behind  when  flying  a  mission. 

My  decision  making  ability  is  as  good  in  emergencies  1 
as  in  routine  mission  situations. 

Leadership  of  the  crew  team  is  solely  the  1 

responsibility  of  the  pilol-in-command. 

Crew  chief  questions  and  suggestions  should  be  1 

considered  by  the  pilots. 

When  joining  a  unit ,  a  new  crewmember  should  not  1 
offer  suggestions  or  opinions  unless  asked. 

The  rank  differences  between  officer  and  enlisted  1 

crewmembers  can  create  barriers  that  threaten 
mission  safety  and  effectiveness. 

Because  crew  chiefs  have  no  pilot  training,  they  1 

should  limit  their  attention  to  their  formally  defined 
crewchief  duties 

Pilots-in-command  who  accept  and  implement  1 

suggestions  from  the  crew  are  lessening  their  stature 
and  reducing  their  authority. 


Crewmembers  should  monitor  the  piiot-in-command's  1 
performance  for  possible  mistakes  and  errors 

Corrections  to  crew  mistakes  should  be  implemented  1 
directly  by  the  pilot-in-command  whenever  physically 
possible. 

The  best  way  to  correct  an  error  is  to  alert  the  error  1 

maker  so  that  he  can  correct  the  problem. 


2  3 

2  3 

2  3 

2  3 

2  3 

2  3 

2  3 

2  3 

2  3 

2  3 

2  3 

2  3 

2  3 

2  3 

2  3 

2  3 


4  5 

4  5 

4  5 

4  5 

4  5 

4  5 

4  5 

4  5 

4  5 

4  5 

4  5 

4  5 

4  5 

4  5 

4  5 

4  5 


Strongly 

Agree  Agree 

6  7 

6  7 

6  7 

6  7 

6  7 

6  7 

6  7 

6  7 

6  7 

6  7 

6  7 

6  7 

6  7 

6  7 

6  7 

6  7 
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Strongly 

Slightly 

Slightly 

Strongly 

Disagree 

Disagree 

Disagree 

Neutral 

Agree 

Agree 

Agree 

31.  Crewmember  errors  and  mistakes  during  the 

mission,  including  the  pilot-in-command's  mistakes, 
should  be  a  significant  part  of  post  flight 
crew  discussions. 

1 

2 

3 

4 

5 

6 

7 

32.  The  pilot-in-command  should  seek  advice  from 
crewmembers  in  updating  mission  plans. 

1 

2 

3 

4 

5 

6 

7 

33.  The  pilot-in-command  should  use  his  crew  to  help  him 
maintain  situation  awareness. 

1 

2 

3 

4 

5 

6 

7 

34.  It  is  solely  the  responsibility  of  the  pilot-in-command  tc 
maintain  awareness  of  crew  capabilities. 

►  1 

2 

3 

4 

5 

6 

7 

35.  Only  when  the  pilot-in-command  is  overloaded  should 
he  pass  workload  to  other  crewmembers. 

1 

2 

3 

4 

5 

6 

7 

36.  Crewmembers  should  be  aware  of  the  workload 
placed  on  other  crewmembers. 

1 

2 

3 

4 

5 

6 

7 

37.  If  a  crewmember  is  having  difficulties  executing  his 
responsibilities,  other  crewmembers  should  provide 
assistance. 

1 

2 

3 

4 

5 

6 

7 

38.  Task  overload  does  not  occur  for  highly  competent 
pilots. 

1 

2 

3 

4 

5 

6 

7 

39.  A  crewmember  should  offer  task  help  to  another 
crewmember  only  if  he  is  sure  the  crewmember 
needs  it. 

1 

2 

3 

4 

5 

6 

7 

40.  A  pilot-in-command  should  not  get  involved  with  the 
execution  of  responsibilities  assigned  to  other 
crewmembers. 

I 

2 

3 

4 

5 

6 

7 

41.  Task  overloads  of  crewmembers  usually  occur 
because  the  overloaded  crewmember  is  not  very 
competent. 

1 

2 

o 

O 

4 

5 

6 

7 

42.  Pilots-in-command  should  employ  the  same 
style  of  management  in  all  situations  and  with 
all  crewmembers. 

1 

2 

3 

4 

5 

6 

7 

43.  Pilot-in-command  instructions  to  other 
crewmembers  should  be  general  and 
non-specific  so  that  each  individual  can  practice 
self-management  and  can  develop  individual 
skills. 

1 

2 

3 

4 

5 

6 

7 

44.  A  relaxed  attitude  is  essential  to  maintaining  a 
cooperative  and  harmonious  cockpit. 

1 

2 

3 

4 

5 

6 

7 

45.  Reprimands  are  more  effective  than  discussions 
in  eliminating  a  poor  flying  habit  in  a  crew- 

1 

o 

4 

5 

6 

7 

member. 
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I.  Flight.  Crew,  and  Equipment  Information 


1.  Date: _ 

2.  Reporting  Time:  _ 

3.  Mission  Total  Flying  Hours: _ 

4.  Mission  Completion  Time:  _ 

5.  Mission  Total  Time: _  (Subtract  item  ^2  from  item  U) 

6.  Type  Equipment:  Acft  Simulator  (circle  one) 

7.  Type  Mission:  SVC  MTF  TRNG  (circle  one) 

Estimated 

8.  NVG  Used:  Y  or  N  (circle  one)  %  Illumination  Predicted:  ____  Actual: 

9.  Mission  Purpose /Description  (include  a  listing  of  ATM  Tasks  Performed  when  appropriate): «« 


10.  Type  Flight  Plan:  VFR  IFR  Composite  (circle  one) 

11.  Predicted  Condition:  VMC  IMC  (circle  one) 

12.  Actual  Condition:  VMC  IMC  (circle  one) 

13.  Crew  Composition  (checkmark  for  each  crewmember  present) 

PC _  PI  _  CP  _  CC  _ _ 


14.  Previous  experience  of  individuals  as  crewmembers  flying  together  regardless  of  previous  seat  position;  for  example,  for 
a  two  person  crew,  one  pair  would  be  marked;  for  a  three  person  crew,  three  pairs  would  be  marked.  (Mark  all  pairings  as 
appropriate.) 

Estimated  Estimated 

Position  Pairing  #  Missions  #  Hours 


a.  PC-PI 
h.  PC-CP 

c.  PC-CC 

d.  PI  -  CP 

e.  PI-CC 
/.  CP-CC 


15.  Cross-Indexing  Code  (Explain  to  aircrew  that  responses  will  not  be  used  to  evaluate  individual  aviators.  Results 
will  not  become  a  part  of  the  aviator's  record.  However,  an  individual  identifier  is  necessary  since  most  aviators 
will  be  completing  other  forms  to  support  the  research  project.) 


a.  PC 

b.  PI 

c.  CP 

d.  CC 


Social  Security  Number 


16.  Evaluator  Name: 


_  17.  Qualification:  IP _ SP _ IE _ ME. 

(Check  One) 
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II.  Crew  Communications  and  Coordination 

(Circle  the  one  number  on  each  dimension  which  best  describes  the  behavior  of  the  crew  during  the  mission. 
Consult  the  'Instructions  for  Making  Ratings  on  the  ACE  (Checklist  Dimensions"  before  making  ratings.) 


CREW  COORDINATION  BEHAVIORS 
!•  Thorough  pre-flight  mission  plan  developed 

2.  Statements/directives  clear,  timely,  relevant, 
complete,  and  verified 

3.  Inquiiy/questioning  practiced 

4.  Advocacy/assertion  practiced 

5.  Decisions  communicated  and  acknowledged 

6.  Actions  communicated  and  acknowledged 

7.  Crew  self-critique  of  decisions  and  actions 

8.  Crewmember  actions  mutually  cross  monitored 

9.  Interpersonal  relationships/group  climate 

10.  Aircraft ,  personnel,  and  mission  status  reported 
11*  Distractions  avoided  or  prioritzed 

12.  Workload  effectively  distributed/redistributed 

13.  Support  informationyactions  sought  from  crew 

14.  Support  information/actions  offered  by  crew 

OVERALL  MISSION  PERFORMANCE  AND  WORKLOAD 

15.  Overall  technical  proficiency 

16.  Overall  crew  effectiveness 

17.  Overall  workload 


1  2  3  4  5 


1  2  3  4  5 


1  2  3  4  5 


1  2  3  4  5 

1  2  3  4  5 

1  2  3  4  5 


1  2  3  4  5 


1  2  3  4  5 


1  2  3 


4  5 


1  2  3  4  5 


6  7 

6  7 

6  7 

6  7 

6  7 

6  7 

6  7 

6  7 

6  7 

6  7 

6  7 

6  7 

6  7 

6  7 

Very 

High 

6  7 

6  7 

6  7 
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ni  Special  Circumstances:  This  section  provides  data  on  non-standard  situations  or  behaviors  that  may 
inHuence  crew  performance.  If  abnormal  emergency  situations  arose,  rate  the  overall  management 
of  the  situation.  If  conflicts  occurred,  rate  how  effectively  they  were  resolved. 


18.  Management  of  abnormal  or  emergency  situation 


4 


A-"' 


/ 


19.  Conflict  resolution 


20. 


Individual  Ratings:  In  some  cases  the  actions  of  a  particular  crewmember  may  be  particularly 
significant  to  the  outcome  of  the  mission.  In  cases  where  this  happens,  enter  the  relevant  item  number 
from  the  above  items  (1-14),  check  the  position  of  the  crewmember  rated,  and  circle  the  appropriate 
number  on  the  dimension  which  reflects  that  individual's  performance. 


PC 

PI 

-  /«. 

CP 

CC 

Item^ 

.  /_ 

.  / 

/ 

PC 

PI 

CP 

CC 

Item. 

.  /_ 

_  /_ 

/ 

PC 

PI 

CP 

CC 

1 

1 

1 


2 

2 

2 


3 

3 

3 


4 

4 

4 


CF 

5 

5 

5 


6 

6 

6 


IV.  Comment  on  any  extreme  or  unusual  (especially  1  or  7)  ratings  on  any  item  in  Section  n  or  HI. 
Item  #  Comments 


7 

7 

7 


V.  Comments  on  Extreme  or  Unusual  Conditions  or  Behaviors:  Describe  conditions,  conflicts,  or 
unusual  individual  behaviors  which  occurred  during  the  mission. 
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VI.  Supplementary  Information:  Conditions  which  significantly  influenced  the  flight  (include  weather, 
ATC  information,  pre-existing  mechanicals,  etc.)  Describe  below. 


VII.  Post  Flight  Questions  (Ask  the  following  questions  of  each  crewmember  after  completion  of  the  flight. 
Record  the  responses  below.) 

1.  Were  you  aware  that  this  specific  mission  or  scenario  would  be  used  prior  to  reporting  to  the 
flight  line  today?  Response  options  are  as  follows: 

0  -  No  Information  about  any  aspect  of  the  mission  or  scenario 

1  -  Slight  Familiarity  with  the  mission  and/or  scenario 

2  -  Considerable  Familiarity  with  the  mission  and/or  scenario 


3  -  Detailed  Information  on  the  mission  and  scenario 


(Circle  one  resp>onse  for  each  participating  crew  member,  (e.g.,  PC: 


)) 


1. 

PC: 

No  Information 

0 

Slight  Farruliarity 

1 

Considerable 

Familiarity 

2 

Detailed 

Information 

3 

2. 

PI: 

0 

1 

2 

3 

3. 

CP: 

0 

1 

2 

3 

4. 

CC: 

0 

I 

2 

3 

2.  To  what  extent  did  you  experience  motion  sickness  during  this  simulator  session /flight? 
(Circle  one  response  for  each  participating  crewmember.) 


None 

Scarcely  any 

Very  Little 

A  little 

Some 

Quite  a  bit 

A  great  deal 

1. 

PC: 

0 

1 

2 

3 

4 

5 

6 

2. 

PI: 

0 

1 

2 

3 

4 

5 

6 

3. 

CP: 

0 

1 

2 

3 

4 

5 

6 

4. 

CC: 

0 

1 

2 

3 

4 

5 

6 
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*  TESTBED  EVALUATION  GRADE  SLIP 


1 

SSN.  PIC: 

AIRCREW 

SSN.  PI: 

EVALUATOR 

NAME: 

TESTBED  SIMULATOR  FLIGHT  DATA 

TIME  TODAY:  ^ _  CUMULATIVE  TIME: 

PIC  _ 

PI  _ 

AS  CREW  _ 

TYPE  SIMULATOR:  2B38  (UH-60  FLIGHT  SIMULATOR) 
SCENARIO  #: 


EVALUATOR  DEBRIEFING  STATEMENT/AIRCREW  GRADE 


A. 3-3 


MANEUVER/PROCEDURE  GRADE  SUP  FOR  TESTBED  AIRCREWS 


PIC 

AIRCREW 

WWlwj  I  IW  — — - - - 

SSN,  PI  - - - - 

EVALUATOR 

NAME: 

_ _ _ _ 

1 

SCENARIO: 

nATF- 

1  !»•  — - - - 

- r - ^ - r - - - r 

NO 


1001 


1002 


1003 


1004 


1005 


1007 


1015 


1016 


1017 


1018 


1019 


1020 


1021 


1022 


1023 


1025 


1026 


maneuver/procedure 

VFBmOHTPlANNWO 


GR* 


IFR  PtiOKT  PUANNINO 


DD  FORM  365^ 


DD  FORM  5701 -R 


PREFUOHT  INSPECTION 


.  engine  START  RUN-UP. AND 
BEFORE  TAKEOFR  CHECKS  ! 


OROyNPTAXI 


HOVER  POWER  CHECK 


HOVERING  FUOHT. 


NORMAL  TAKEOFF 


ROLUNQ  TAKEOFF 


SIMULATED  MAXIMUM 
PERFORMANCE  TAKEOFF 


DECELERATION/ACCELERATION 


TRAFFIC  PATTERN  FUOHT 


FUEL  MANAGEMENT  PROCEDURES 


PILOTAGE  AND  DEAD  RECKONING 


DOPPLER  NAVIGATION 


1027  BEFORE-LANDING  CHECK 


1029 


maneuver/procedure  I  OR 

VMC  APPROACH 


ROLL-ON  LANDING 


1031 


CONFINED  AREA  OPERATIONS. 


1032 


1036 


1053 


1057 


1056 


1062 


1063 


1066 


1071 


SLOPE  OPERATIONS 


HOVER  OGE  CHECK 


SIMULATED  ENGINE  FA^Ure  AT 

ALTity^V/X 


DEGRADED  AFCS 


ECU  LOCKOUT  OPERATIONS 


STABILATOR  MALFUNCTION 
PROCEDURES 


EMERGENCY  PROCEDURES 


AIRCREW  COORDINATION 


1075 


1076 


1077 


INSTRUMENT  TAKEOFF 
RADIO  NAVIGATION 
HOLDING  PROCEDURES 


1078  UNUSAL  ATTITUDE  RECOVERY 


1079 


1080 


RADIO  COMMUNICATIONS 
PROCEDURES 

PROCEDURES  FOR  TWO-WAY 
RADIO  FAILURE 


SIMULATED  HYORAUUC  SYSTEM 
MALFUNCTION 


MANEUVER/PROCEDURE  GRADE  SLIP  FOR  TESTBED  AIRCREWS 


(CONTINUED) 


NO  MANEUVER/PROCEDURE  GR* 


1081  NONPRECISION  APPROACH 


1082  PRECISION  APPROACH 


1083  VHIRP 


1084  COMMAND  INSTRUMENT 
SYSTEM  OPERATIONS 


1096  AIRCRAFT  SURVIVABIUTY 


1098  ixAFtER^NbiNOtAE^S:: 


MARK  XII  IFF  SYSTEM 


PINNACLE  OR  RIOOEUNE 
OPERATION 


FM  RADIO  HOMING 


AERIAL  OBSERVATION 


2008  EVASIVE  MANEUVERS  ' 


2009  l  .i.lMLTIAIRCRArr.^EFM^ON^ 


2010  RAPPELUNO  OPERATIONS 


INTERNAL  RESCUE-HOIST 
OPERATIONS 


20^2  aerial  MINE  DEUVERY 


::|*EREbrtME)CtERNAll.OAD 


xPEWPRMTWRAiHl'MGHT; 


i.NRf'bftWtERliWiMI^UcJHt: 


For  highlighted  tasks,  top  half  of 
grade  Diock  is  for  maneuver/ 

grocedure  grade,  if  grade  Is 
.  C,  or  U,  enter  If  flight  skill 
T  or  aircrew  coordination  'a' 
deficiency.  For  'a*,  note  standard 
from  Task  1071.  (See  example 
helow) 


TESTBED  EVALUATION  COMMENT  SUP 


_ 

SSN.  PIC 

AIRCREW 

SSN,  PI 

EVALUATOR 

NAME: 

SCENARIO: 

DATE: 

COMMENTS 
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Experimental  Ratings  of  Aviator  Qualities 


A.  4-2 


Experimental  Rating  of  Aviator  Qualities 


i 


The  Army  Research  Institute  (ARI)  is  researching  the  area  of  cockpit 
management  in  Army  Aviation.  The  goal  is  to  improve  performance  and  increase 
the  margin  of  safety  on  an  Army-wide  basis.  Other  DoD  services  and 
commercial  aviation  have  also  looked  into  the  area  of  cockpit  management  and 
have  realized  substantial  gains  in  performance  and  safety.  Army  Aviation  is 
unique  and  much  of  what  has  been  discovered  in  other  service  branches  and  the 
commercial  world  is  not  applicable  to  the  Army. 

Consequently,  ARI's  program,  designed  to  meet  the  needs  of  the  Army,  is 
multifaceted.  Simulations  are  being  developed  to  stress  crew-type  tasks;  it  is 
envisioned  that  enhanced  trainmg  will  be  developed;  the  US  Army  Safety  Center 
is  incorporating  crew  coordination  errors  into  their  investigation  process;  and 
revisions  to  the  APARTS  program  are  being  planned. 

One  component  of  ARI's  research  program  is  the  Army  Aviation  Crewmember 
Questionnaire  that  asks  aviators  about  their  attitudes  towards  cockpit 
management.  Aviators  in  your  unit  are  being  administered  the  questionnaire. 
There  is,  however,  some  additional  information  that  we  require  to  supplement 
the  questionnaire  data.  Your  assistance  in  providing  this  additional  information  is 
essential  to  the  success  of  the  research  program  since  results  of  this  survey  will 
be  used  to  guide  us  into  our  next  phase. 


IMPORTANT 


The  information  you  provide  in  this  questionnaire  is  off  the  record  and  will  be  used  for 
research  purposes  only.  It  will  be  kept  completely  confidential.  It  will  not  be  attributed  to 
you  personally,  nor  will  it  become  a  part  of  any  records  kept  on  aviators  in  your  unit.  The 
names  on  the  questionnaire  are  there  oniy  for  your  convenience  and  will  be  removed  from 
the  questionnaire  once  it  is  obtained  by  us.  The  sociai  security  numbers  on  the 
questionnaire  wiii  be  used  oniy  as  a  cross-index  number  within  the  research  database. 


Rating  Directions 


Use  the  attached  sheets  for  your  rating.  Go  to  the  sheet  marked  "Rating  Form-Aviator  Qualities"  and 
start  by  filling  out  the  names  of  aviators  in  your  unit  with  whom  you  are  familiar.  Use  continuation 
sheets  if  necessary.  Choose  only  those  aviators  whose  habits,  style,  skill  level,  etc.  you  are  familiar 
with.  You  will  need  to  get  the  aviators'  social  security  number  from  their  records  or  ask  them  directly. 

Using  a  three  level  scale,  you  will  rate  the  aviators  in  each  one  of  the  four  categories  (columns) 
marked  Cockpit  Resource  Management,  Flying  Skills,  Safety,  and  Mission  Effectiveness.  You  will 
give  a  rating  of  1  to  an  aviator  in  the  top  25%,  a  rating  of  2  to  an  aviator  iri  the  middle  50%,  and  a 
rating  of  3  to  an  aviator  in  the  bottom  25%  of  the  group.  Each  individual  aviator  may  be  rated  in  the 
same  percentile  bracket  or  a  different  percentile  bracket  for  each  major  area.  The  rating  you  give  an 
aviator  in  one  category  has  no  bearing  on  how  you  rate  the  aviator  in  another  category. 


Constructing  the  Scale 

You  will  use  the  same  three  level  scale  for  each  category.  You  will  need  to  construct  the  scale 
yourself.  Here's  how.  First,  total  the  number  of  aviators  on  your  sheet.  Next,  divide  the  total  by  four. 
The  number  resulting  from  the  division  (without  the  remainder)  is  the  number  of  aviators  you  will  give 
a  1  rating.  It  is  also  the  number  of  aviators  you  will  give  a  3  rating.  The  rest  of  the  aviators  will  be 
rated  as  a  2. 

For  example,  say  there  are  14  aviators  you  are  rating  on  your  sheet.  You  divide  14  by  4,  and  the 
answer  is  3  (disregard  the  remainder  of  2).  Therefore,  you  will  give  three  aviators  a  rating  of  1 ,  eight 
aviators  a  rating  of  2,  and  three  aviators  a  rating  of  3. 

Steps: 

Step  1 :  Total  the  number  of  aviators  you  will  rate. 

Step  2:  Divide  the  total  by  4. 

Step  3:  Use  the  quotient  (without  the  remainder)  as  the  number  of 
1  ratings  and  the  number  of  3  ratings  you  can  allocate. 

Step  4:  The  rest  of  the  aviators  are  allocated  a  rating  of  2 


EXAMPLES 

Example  1 : 

Step  1 

There  are  14  aviators  with  whom  you  are  familiar. 

Step  2 

14/4  =  3,  remainder  2 

Step  3 

Allocation  Allowance  Is:  three  1  ratings 

&4 

eight  2  ratings 
three  3  ratings 

Example  2: 

Step  1 

There  are  17  aviators  with  whom  you  are  familiar. 

Step  2 

17/4  =  4,  remainder  1 

Step  3 

Allocation  Allowance  is:  four  1  ratings 

&4 

nine  2  ratings 
four  3  ratings 

Once  you  know  how  many  aviators  you  can  place  within  each  of  the  three  levels,  proceed  to  apply 
the  ratings  in  the  columns.  The  definitions  to  be  used  in  considering  each  column  are  on  the  next 
page.  A.  4-4 


Applying  the  Ratings 


Within  each  category  (coiumn)  give  the  aviators  a  rating  of  1,  2,  or  3.  Use  exactly  the  number 
of  1  's,  2's  and  3's  you  are  allowed  within  each  category  (column).  Descriptions  of  the  qualities 
named  at  the  top  of  each  column  are  given  below.  Each  column  is  for  a  distinct  quality.  You'll 
need  to  read  the  descriptions  carefully  so  you  understand  how  each  column  requires  a  rating 
of  a  different  quality. 

The  aviators  on  your  list  are  probably  of  varying  experience  levels.  Try  to  rate  the  aviators 
taking  into  account  their  experience  levels.  For  example,  a  new  aviator  may  not  be  one  of  your 
best  cockpit  resource  managers,  but  considering  his  experience  level  he  may  be  among  the 
top  25%  of  aviators  at  his  experience  level.  In  that  case,  you  should  give  him  a  rating  of  1 . 


Cockpit  Resource  Management  -  The  effective  cockpit  resource  manager  attempts  to 
establish  and  maintain  positive  working  and  interpersonal  relationships  to  create  a  harmonious 
team  atmosphere  and  to  execute  mission  objectives.  He  is  sensitive  to  the  capabilities  of  his 
fellow  crewmembers,  and,  while  he  is  a  good  leader,  does  not  lead  based  on  rank  or  crew 
position  alone.  He  understands  that  errors  are  a  fact  of  life  and  checks  other's  performance  to 
detect  errors.  He  manages  workload  well,  and,  in  concert  with  the  crew,  effectively 
redistributes  workload  as  the  mission  proceeds.  He  voluntarily  helps  out  whenever  he  can.  He 
maintains  situation  awareness  and  helps  to  prioritize  crew  tasks  to  ensure  that  the  aircraft  is 
being  operated  within  acceptable  parameters  and  that  appropriate  clearances  are  maintained. 
Finally,  the  effective  cockpit  manager  is  a  good  communicator;  his  communications  are  clear, 
timely,  complete,  relevant  and  transmitted  using  standard  terminology.  He  seeks 
acknowledgement  of  his  transmissions  and,  likewise,  verifies  receipt  when  others  direct  or 
provide  information  to  him. 


Flying  Skills  -  This  rating  is  an  estimation  of  the  aviator's  flying  proficiency  -  his  "stick  and 
rudder"  skills.  This  aviator,  regardless  of  his  experience  level,  is  in  tune  with  the  aircraft.  While 
staying  within  acceptable  parameters,  he  generally  obtains  optimal  aircraft  performance.  He  is 
a  capable  tactical  aviator,  understands  what  needs  to  be  done  in  emergency  or  abnormal 
conditions,  and  knows  how  to  handle  the  aircraft  in  even  the  most  difficult  situations. 


Safety  -  This  rating  reflects  the  degree  of  safety  awareness  demonstrated  by  the  aviator.  A 
safe  aviator,  while  willing  to  take  risks,  can  assess  risks  in  relation  to  mission  objectives  and 
individual  capabilities.  In  accepting  a  particular  level  of  risk,  the  aviator  always  balances  the 
safety  of  the  aircrew  and  the  aircraft  against  accomplishing  the  mission.  While  the  aviator  may 
be  strongly  mission  oriented,  unacceptable  risk  is  always  rejected. 


Mission  Effectiveness-  This  rating  is  more  global  than  the  three  mentioned  above.  It  takes 
into  account  all  three  qualities  and  how  the  pilot  combines  them  to  accomplish  assigned 
missions.  The  mission  effective  pilot  "gets  the  job  done."  He  knows  how  to  manage  cockpit 
resources,  maintains  safety  awareness,  and  has  highly  regarded  flying  and  navigation  skills. 
He  is  the  one  to  whom  you  would  trust  the  most  difficult  assignments  and  missions  since  he 
has  the  right  mix  of  intelligence,  skills,  attitudes,  and  courage  to  succeed. 
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APPENDIX  B 


CMAQ  FREQUENCY  TABLES 
Table  #1 

Cl  Crewmembers  should  avoid  disagreeing  with  others  because  conflicts 
create  tension  and  reduce  crew  effectiveness. 


Value 

Frequency 

Percent 

Strongly 

Disagree 

1 

17 

10.1 

Disagree 

2 

64 

38.1 

Slightly 

Disagree 

3 

30 

17.9 

Neutral 

4 

9 

5.4 

Slightly 

Agree 

5 

20 

11.9 

Agree 

6 

25 

14.9 

Strongly 

Agree 

7 

3 

1.8 

TOTAL 

168 

100.0 

Mean 

3.226 

Median 

3.000 

Std 

Valid  Cases  168  Missing  Cases  0 


1.705 


Table  #2 

C2  Crewmembers  should  feel  obligated  to  mention  their  own 
psychological  stress  or  physical  problems  to  other  crewmembers 
before  or  during  a  mission. 


Disagree 

Slightly  Disagree 
Neutral 

Slightly  Agree 
Agree 

Strongly  Agree 


Value  Frequency  Percent 


2  6  3.6 

3  5  3.0 

4  10  6.0 

5  28  16.7 

6  76  45.2 

7  43  25.6 


TOTAL 

Median 

Missing  Cases 


168  100.0 

6.000 

0 


Mean 

Valid  Cases 


5.738 

168 


Std  Dev 


1.200 


Table  #3 


It  is  important  to  comment  about  the  procedures  and  techniques  of 
other  crewmembers . 


Disagree 

Slightly  Disagree 
Neutral 

Slightly  Agree 
Agree 

Strongly  Agree 


Mean  5.536 

Valid  Cases  168 


Value  Frequency 


2  4 

3  8 

4  10 

5  37 

6  90 

7  19 

TOTAL  168 

Median  6.000 

Missing  Cases  0 


Percent 

2.4 
4.8 
6.0 
22.0 
53 . 6 
11.3 

100.0 

Std  Dev  1.083 


Table  #4 


Pilots-in-command  should  not  dictate  flight  techniques  to  other 
crewmembers . 


Value 

Frequency 

Percent 

Strongly 

Disagree 

1 

4 

2.4 

Disagree 

2 

24 

14.3 

Slightly 

Disagree 

3 

29 

17.3 

Neutral 

4 

13 

7.7 

Slightly 

Agree 

5 

39 

23.2 

Agree 

6 

49 

29.2 

Strongly 

Agree 

7 

10 

6.0 

TOTAL 

168 

100.0 

Mean 

4.464 

Median 

5.000 

Std 

Valid  Cases  168  Missing  Cases  0 


1.641 
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Table  #5 


C5  Casual  social  conversation  during  periods  of  low  workload  can 
improve  crew  coordination. 


Value  Frequency  Percent 

2  3  1.8 

3  5  3.0 

4  24  14.3 

5  32  19.0 

6  78  46.4 

7  26  15.5 

TOTAL  168  100.0 

Mean  5.518  Median  6.000  Std  Dev  1.116 

Valid  Cases  168  Missing  Cases  0 


Disagree 

Slightly  Disagree 
Neutral 

Slightly  Agree 
Agree 

Strongly  Agree 


Table  #6 

Each  crewmember  should  monitor  other  crewmembers  for  signs  of 
stress  or  fatigue,  and  should  discuss  the  situation  with  the 
crewmember . 


Value  Frequency  Percent 


Neutral 

Slightly  Agree 
Agree 

Strongly  Agree 


Mean  6.214 

Valid  Cases  168 


4  3 

5  16 

6  91 

7  58 

TOTAL  168 

Median  6.000 

Missing  Cases  0 


1.8 

9.5 

54.2 

34.5 

100.0 

Std  Dev  .685 


Table  #7 


C7  Good  coimunications  and  crew  coordination  are  as  important  as 
technical  proficiency  for  the  safety  of  the  flight. 


Slightly  Disagree 
Neutral 

Slightly  Agree 
Agree 

Strongly  Agree 


Mean  6.500 

Valid  Cases  168 


Value 

Frequency 

Percent 

3 

1 

.  6 

4 

2 

1.2 

5 

7 

4.2 

6 

60 

35.7 

7 

98 

58.3 

TOTAL 

168 

100.0 

Median 
Missing  < 

7.000 
Cases  0 

Std 

.  692 


Table  #8 

C8  Crewmembers  should  be  aware  of  and  sensitive  to  the  personal 
problems  of  other  crewmembers. 


Disagree 

Slightly  Disagree 
Neutral 

Slightly  Agree 
Agree 

Strongly  Agree 


Mean  5.756 

Valid  Cases  168 


Value  Frequency 


2  1 

3  1 

4  10 

5  39 

6  92 

7  25 

TOTAL  168 

Median  6.000 

Missing  Cases  0 


Percent 

.6 

.6 

6.0 

23.2 

54.8 

14.9 

100.0 

Std  Dev  .844 
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Table  #9 


C9  The  pilot-in-cominand,  time  and  situation  permitting,  should  take 
control  and  fly  the  aircraft  in  all  emergency  and  non-standard 
situations. 


Strongly  Disagree 
Disagree 

Slightly  Disagree 
Neutral 

Slightly  Agree 
Agree 

Strongly  Agree 


Mean  3.696 

Valid  Cases  168 


Value  Frequency 


1  15 

2  47 

3  26 

4  16 

5  22 

6  34 

7  8 

TOTAL  168 

Median  3.000 

Missing  Cases  0 


Percent 

8.9 

28.0 

15.5 

9.5 

13.1 

20.2 
4.8 

100.0 

Std  Dev  1.837 


Table  #10 

CIO  The  pilot  flying  the  aircraft  should  verbalize  plans  for  procedures 
or  maneuvers  and  should  be  sure  the  information  is  understood  and 
acknowledged  by  crewmembers  affected. 


Value 

Frequency 

Percent 

Slightly 

Disagree 

3 

1 

.6 

Neutral 

4 

3 

1*8 

Slightly 

Agree 

5 

20 

11.9 

Agree 

6 

97 

57.7 

Strongly 

Agree 

7 

47 

28.0 

TOTAL 

168 

100.0 

Mean 

6.107 

Median 

6.000 

Std 

Valid  Cases  168 

Missing  ' 

Cases  0 
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Table  #11 


Cll  Pilots  and  other  crewmembers  should  not  question  the  decisions  or 
actions  of  the  pilot-in-command  except  when  these  actions  obviously 
threaten  the  safety  of  the  flight. 


Value 

Frequency 

Percent 

Strongly  Disagree 

1 

14 

8.3 

Disagree 

2 

67 

39.9 

Slightly  Disagree 

3 

34 

20.2 

Neutral 

4 

9 

5.4 

Slightly  Agree 

5 

22 

13.1 

Agree 

6 

18 

10.7 

Strongly  Agree 

7 

TOTAL 

4 

168 

2.4 

100.0 

Mean  3.167  Median  3.000  Std  Dev  1.626 

Valid  Cases  168  Missing  Cases  0 


Table  #12 

C12  Even  when  fatigued,  I  perform  effectively  during  most  critical 
flight  maneuvers. 


Value 

Frequency 

Percent 

Strongly 

Disagree 

1 

15 

8.9 

Disagree 

2 

50  ' 

29.8 

Slightly 

Disagree 

3 

32 

19.0 

Neutral 

4 

21 

12.5 

Slightly 

Agree 

5 

28 

16.7 

Agree 

6 

20 

11.9 

Strongly 

Agree 

7 

2 

1.2 

TOTAL  168  100.0 

Median  3.000  Std  Dev  1.604 

Missing  Cases  0 


Mean  3.387 

Valid  Cases  168 
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C13 


Table  #13 

PUots-in-coim.and  should  encourage 
bastion  procedures  and  flight 
flight  operations  and  in  emergenc 


Value 

Frequency 

Percent 

1 

1 

.6 

Strongly  Disagree 

2 

9 

5.4 

Disagree 

3 

7 

4.2 

Slightly  Disagree 

4 

6 

3.6 

Neutral 

5 

44 

26.2 

Slightly  Agree 

6 

77 

45.8 

Agree 

7 

24 

14.3 

strongly  Agree 

TOTAL 

168 

100.0 

a,  5.440 

Median 

6.000 

Std 

Mean  ico 

Valid  Cases  los 

Missing 

Cases  0 

C14 


Table  #14 


lere  are  no  circumstances  where  Jhe  pilot 
mtrols  without  being  directed  to  do  so 


should  take  the  aircraft 
by  the  pilot-in-command. 


Value  Frequency  Percent 


Strongly  Disagree 

1 

43 

25.6 

2 

77 

45.8 

Disagree 

3 

25 

14.9 

Slightly  Disagree 

4 

7 

4.2 

Neutral 

5 

3 

1.8 

Slightly  Agree 

6 

12 

7.1 

Agree 

7 

1 

.6 

Strongly  Agree 

TOTAL 

168 

100.0 

Mean  2.345 

Valid  Cases  168 

Median 

Missing  Cases 

2.000 

0 

Std 
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Table  #15 


C15  A  debriefing  and  critique  of  procedures  and  decisions 
mission  is  an  important  part  of  developing  and 
effective  crew  coordination. 


Neutral 

Slightly  Agree 
Agree 

Strongly  Agree 


Mean  6.333 

Valid  Cases  168 


Value  Frequency  Percent 


4 

3 

1.8 

5 

12 

7.1 

6 

79 

47.0 

7 

74 

44.0 

TOTAL 

168 

100.0 

Median 

6.000 

Std 

Missing  Cases  0 


Table  #16 

C16  Training  is  one  of  the  pilot-in-command  •  s 

responsibilities . 


Strongly  Disagree 
Disagree 

Slightly  Disagree 
Neutral 

Slightly  Agree 
Agree 

Strongly  Agree 


Mean  6.119 

Valid  Cases  168 


Value 

Frequency 

Percent 

1 

1 

.6 

2 

1 

.6 

3 

3 

1.8 

4 

5 

3 . 0 

5 

13 

7.7 

6 

84 

50.0 

7 

61 

36.3 

TOTAL 

168 

100.0 

Median 

6.000 

Std 

Missing  Cases  0 


after  each 
maintaining 


.689 


important 


.978 
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Table  #17 


C17  Under  high  stress,  good  crew  coordination  is  more  important  than  it 
is  under  low  stress  conditions. 


Strongly  Disagree 
Disagree 

Slightly  Disagree 
Neutral 

Slightly  Agree 
Agree 

Strongly  Agree 


Mean  5.452 

Valid  Cases  168 


Value 

Frequency 

Percent 

1 

6 

3.6 

2 

13 

7.7 

3 

12 

7.1 

4 

4 

2.4 

5 

17 

10.1 

6 

65 

38.7 

7 

51 

30.4 

TOTAL 

168 

100.0 

Median 
Missing  ' 

6.000 
Cases  0 

Std 

1.730 


Table  #18 

C18  Effective  crew  coordination  requires  crewmembers  to  take  into 
account  the  personalities  of  other  crewmembers . 


Strongly  Disagree 
Disagree 

Slightly  Disagree 
Neutral 

Slightly  Agree 
Agree 

Strongly  Agree 


Mean  5.732 

Valid  Cases  168 


Value  Frequency 


1  1 

2  3 

3  2 

4  9 

5  37 

6  83 

7  33 

TOTAL  168 

Median  6.000 

Missing  Cases  0 


Percent 

.  6 

1.8 

1.2 

5.4 

22.0 

49.4 

19.6 

100.0 

Std  Dev  1.052 
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Table  #19 


C19  The  pilot-in-command' s  responsibilities  include  coordination  of 
inflight  crew  chief  activities. 


Neutral 

Slightly  Agree 
Agree 

Strongly  Agree 


Mean  6.161 

Valid  Cases  168 


Value  Frequency 


4  3 

5  14 

6  104 

7  47 

TOTAL  168 

Median  6.000 

Missing  Cases  0 


Percent 

1.8 

8.3 

61.9 

28.0 

100.0 

Std  Dev  .641 


Table  #20 

C20  Most  crewmembers  can  leave  personal  problems  behind  when  flying  a 
mission. 


Strongly  Disagree 
Disagree 

Slightly  Disagree 
Neutral 

Slightly  Agree 
Agree 

Strongly  Agree 


Mean  3.595 

Valid  Cases  168 


Value  Frequency 


1  12 

2  42 

3  34 

4  23 

5  30 

6  25 

7  2 

TOTAL  168 

Median  3.000 

Missing  Cases  0 


Percent 

7.1 
25.0 
20.2 
13.7 

17.9 

14.9 

1.2 

100.0 

Std  Dev  1.606 
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Table  #21 

C21  My  decision  making  ability  is  as  good  in  emergencies  as  in  routine 
mission  situations. 


Strongly  Disagree 
Disagree 

Slightly  Disagree 
Neutral 

Slightly  Agree 
Agree 

Strongly  Agree 


Mean  4.542 

Valid  Cases  168 


Value  Frequency 


1  5 

2  15 

3  31 

4  22 

5  30 

6  58 

7  7 

TOTAL  168 

Median  5.000 

Missing  Cases  0 


Percent 

3.0 

8.9 

18.5 
13.1 
17.9 

34.5 
4.2 

100.0 

Std  Dev  1.570 


Table  #22 

C22  Leadership  of  the  crew  team  is  solely  the  responsibility  of  the 
pi lot- in-command . 


Strongly  Disagree 
Disagree 

Slightly  Disagree 
Neutral 

Slightly  Agree 
Agree 

Strongly  Agree 


Value 

Frequency 

Percent 

1 

13 

7.7 

2 

47 

28.0 

3 

27 

16.1 

4 

15 

8.9 

5 

28 

16.7 

6 

21 

12.5 

7 

17 

10.1 

TOTAL 

168 

100.0 

Median 

Missing 

3.000 
Cases  0 

Std 

Mean 

Valid  Cases 


3.768 

168 


1.876 


Table  #23 


questions  and  suggestions  should  be  considered  by  the 


Strongly  Disagree 
Disagree 
Slightly  Agree 
Agree 

Strongly  Agree 


Mean  6.274 

Valid  Cases  168 


Value  Frequency 


1  1 

2  1 

5  12 

6  87 

7  67 

TOTAL  168 

Median  6.000 

Missing  Cases  0 


Percent 

.6 

.6 

7.1 

51.8 

39.9 

100.0 

Std  Dev  .802 


Table  #24 


C24 


When  joining  a  unit 
or  opinions  unless 


,  a  new  crewmember  should  not 
asked. 


offer 


suggestions 


Strongly  Disagree 
Disagree 

Slightly  Disagree 
Neutral 

Slightly  Agree 
Agree 

Strongly  Agree 


Mean  2.065 

Valid  Cases  168 


Value 

Frequency 

Percent 

1 

48 

28.6 

2 

84 

50.0 

3 

23 

13.7 

4 

7 

4.2 

5 

3 

1.8 

6 

2 

1.2 

7 

1 

.6 

TOTAL 

168 

100.0 

Median  2.000 

Missing  Cases  0 

std 

1.045 
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Table  #25 


C25  The  rank  differences  between  officer  and  enlisted  crewmembers  can 
create  barriers  that  threaten  mission  safety  and  effectiveness. 


Value 

Frequency 

Percent 

Strongly 

Disagree 

1 

18 

10.7 

Disagree 

2 

49 

29.2 

Slightly 

Disagree 

3 

21 

12.5 

Neutral 

4 

11 

6.5 

Slightly 

Agree 

5 

33 

19.6 

Agree 

6 

32 

19.0 

Strongly 

Agree 

7 

4 

2.4 

TOTAL 

168 

100.0 

Mean 

3.619 

Median 

3.000 

Std 

Valid  Cases  168  Missing  Cases  0 


1.817 


Table  #26 

C26  Because  crew  chiefs  have  no  pilot  training,  they  should  limit  their 
attention  to  their  formally  defined  crewchief  duties. 


Value 

Frequency 

Percent 

Strongly  Disagree 

1 

32 

19.0 

Disagree 

2 

69 

41.1 

Slightly  Disagree 

3 

42 

25.0 

Neutral 

4 

12 

7.1 

Slightly  Agree 

5 

6 

3.6 

Agree 

6 

4 

2.4 

Strongly  Agree 

7 

3 

1.8 

TOTAL 

168 

100.0 

Mean  2.494 

Median 

2.000 

Std 

Valid  Cases  168  Missing  Cases  0 


1.281 
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Table  #27 


C27  Pilots-in-coimand  who  accept  and  implement  suggestions  from  the 
crew  are  lessening  their  stature  and  reducing  their  authority. 


Value  Frequency  Percent 


Strongly 

Disagree 

1 

76 

45.2 

Disagree 

2 

74 

44.0 

Slightly 

Disagree 

3 

13 

7.7 

Neutral 

4 

,  3 

1.8 

Slightly 

Agree 

5 

1 

.6 

Agree 

6 

1 

.6 

TOTAL 

168 

100.0 

Mean 

1.702 

Median 

2.000 

Std  Dev 

Valid  Cases  168 

Missing  Cases 

0 

Table  #28 

C28  Crewmembers  should  monitor  the  pilot-in-command's  performance  for 
possible  mistakes  and  errors. 


Value  Frequency  Percent 


Strongly  Disagree 

1 

3 

1.8 

Disagree 

2 

10 

6.0 

Slightly  Disagree 

3 

9 

5.4 

Neutral 

4 

15 

8.9 

Slightly  Agree 

5 

33 

19.6 

Agree 

6 

74 

44.0 

Strongly  Agree 

7 

24 

14.3 

TOTAL 

168 

100.0 

Mean  5.280 

Median 

6.000 

Std 

Valid  Cases  168 

Missing  Cases 

0 
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Table  #29 


C29  Corrections  to  crew  mistakes  should  be  implemented  directly  by  the 
pilot-in-command  whenever  physically  possible. 


Strongly  Disagree 
Disagree 

Slightly  Disagree 
Neutral 

Slightly  Agree 
Agree 

Strongly  Agree 


Mean  4.565 

Valid  Cases  168 


Value  Frequency  Percent 


1  1 

2  24 

3  15 

4  31 

5  36 

6  58 

7  3 

TOTAL  168 

Median  5.000 

Missing  Cases  0 


.  6 

14.3 
8.9 

18.5 

21.4 

34.5 
1.8 

100.0 

Std  Dev 


1.471 


Table  #30 

C30  The  best  way  to  correct  an  error  is  to  alert  the  error  maker  so 
that  he  can  correct  the  problem. 


Disagree 

Neutral 

Slightly  Agree 
Agree 

Strongly  Agree 


Value  Frequency  Percent 


2  4  2.4 

4  5  3.0 

5  32  19.0 

6  95  56.5 

7  32  19.0 


TOTAL  168  100.0 


Median  6.000 

Missing  Cases  0 


Mean 

Valid  Cases 


5.845 

168 


Std  Dev 


.929 


Table  #31 


C31  Crewmember  errors  and  mistakes  during  the  mission,  including  the 
pilot-in-command's  mistakes,  should  be  a  significant  part  of  post 
flight  crew  discussions. 


Value  Frequency  Percent 


Disagree 

Slightly 

Neutral 

Slightly 

Agree 

Strongly 

Disagree 

Agree 

Agree 

2 

3 

4 

5 

6 

7 

TOTAL 

1 

3 

1 

20 

90 

53 

168 

.6 

1.8 

.6 

11.9 

53.6 

31.5 

100.0 

Mean  6.107 

Valid  Cases  168 

Median 

Missing  Cases 

6.000 

0 

Std  Dev 

.841 

Table  #32 

C32  The  pi lot- in-command  should 
updating  mission  plans. 

seek  advice  from 

crewmembers  in 

Value 

Frequency 

Percent 

Disagree 

Slightly 

Neutral 

Slightly 

Agree 

Strongly 

Disagree 

Agree 

Agree 

2 

3 

4 

5 

6 

7 

TOTAL 

1 

2 

3 

33 

94 

35 

168 

.  6 

1.2 

1.8 

19 . 6 
56.0 
20.8 

100.0 

Mean  5.917 

Valid  Cases  168 

Median  6.000 

Missing  Cases  0 

Std 

Dev  .822 

i 


Table  #33 

C33  The  pi  lot- in-command  should  use  his  crew  to  help  him  maintain 
situation  awareness. 


Slightly  Agree 
Agree 

Strongly  Agree 


Mean 

Valid  Cases 


Value  Frequency  Percent 


5 

6 
7 

TOTAL 


6.488  Median 

168  Missing  Cases 


6 

74 

88 

168 

7.000 


3.6 

44.0 

52.4 

100.0 

Std  Dev 


569 


Table  #34 

C34  It  is  solely  the  responsibility  of  the  pilot— in— command  to 
maintain  awareness  of  crew  capabilities. 


Strongly  Disagree 
Disagree 

Slightly  Disagree 
Neutral 

Slightly  Agree 
Agree 

Strongly  Agree 


Mean  2.874 

Valid  Cases  167 


Value 

Frequency 

Percent 

1 

27 

16.1 

2 

68 

40.5 

3 

27 

16.1 

4 

13 

7.7 

5 

11 

6.5 

6 

18 

10.7 

7 

3 

1.8 

• 

1 

.6 

TOTAL 

168 

100.0 

Median 
Missing  - 

2.000 
Cases  1 

Std 

1.629 
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Table  #35 


C35  Only  when  the  pilot-in-command  is  overloaded  should  he 
workload  to  other  crewmembers. 


Strongly  Disagree 
Disagree 

Slightly  Disagree 
Neutral 

Slightly  Agree 
Agree 

Strongly  Agree 


Mean  1.929 

Valid  Cases  168 


Value 

Frequency 

Percent 

1 

56 

33.3 

2 

88 

52.4 

3 

15 

8.9 

4 

3 

1.8 

5 

2 

1.2 

6 

3 

1.8 

7 

1 

.6 

TOTAL 

168 

100.0 

Median 

2.000 

Std 

Missing  Cases  0 


Table  #36 

C36  Crewmembers  should  be  aware  of  the  workload  placed  on 
crewmembers . 


Value  Frequency  Percent 


Neutral 

4 

1 

.6 

Slightly 

Agree 

5 

13 

7.7 

Agree 

6 

115 

68.5 

Strongly 

Agree 

7 

39 

23.2 

TOTAL 

168 

100.0 

Mean 

6.143 

Median 

6.000 

Std 

Valid  Cases  168 

Missing  Cases 

0 

pass 


.018 


other 


.561 
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Table  #37 

Q37  jf  a  crewmember  is  having  difficulties^  executing 

responsibilities,  other  crewmembers  should  provide  assistance 


Disagree 
Slightly  Agree 
Agree 

Strongly  Agree 


Mean  6.214 

Valid  Cases  168 


Value 

Frequency 

Percent 

2 

1 

.6 

5 

11 

6.5 

6 

105 

62.5 

7 

51 

30.4 

TOTAL 

168 

100.0 

Median 

6.000 

Std 

Missing  Cases  0 


Table  #38 

C38  Task  overload  does  not  occur  for  highly  competent  pilots. 


Strongly  Disagree 
Disagree 

Slightly  Disagree 
Neutral 

Slightly  Agree 
Agree 

Strongly  Agree 


Mean  1.810 

Valid  Cases  168 


Value  Frequency 


1  70 

2  78 

3  12 

4  3 

5  1 

6  3 

7  1 

TOTAL  168 

Median  2.000 

Missing  Cases  0 


Percent 

41.7 

46.4 

7.1 

1.8 

.6 

1.8 

.6 

100.0 

Std  Dev  1 


his 


649 


009 
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Table  #39 


C39  A  crewmember  should  offer  task  help  to  another  crewmember  only  if 
he  is  sure  the  crewmember  needs  it. 


Value  Frequency  Percent 


Strongly  Disagree 

1 

17 

10.1 

Disagree 

2 

67 

39.9 

Slightly  Disagree 

3 

54 

32.1 

Neutral 

4 

13 

7.7 

Slightly  Agree 

5 

9 

5.4 

Agree 

6 

8 

4.8 

TOTAL 

168 

100.0 

Mean  2.726 

Median 

2.500 

Std  Dev 

Valid  Cases  168 

Missing  Cases 

0 

Table  #40 

C40  A  pilot-in-co^and  should  not  get  involved  with  the  execution  of 
responsibilities  assigned  to  other  crewmembers. 


Value  Frequency  Percent 

1  21  12.5 

2  86  51.2 

3  36  21.4 

4  9  5.4 

5  11  6.5 

6  2  1.2 

7  3  1.8 

TOTAL  168  100.0 

Mean  2.530  Median  2.000  Std  Dev  1.228 

Valid  Cases  168  Missing  Cases  0 


Strongly  Disagree 
Disagree 

Slightly  Disagree 
Neutral 

Slightly  Agree 
Agree 

Strongly  Agree 
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Table  #41 

C41  Task  overloads  of  crewmembers  usually  occur  because  the  overloaded 
crewmember  is  not  very  competent. 


Value  Frequency  Percent 


Strongly  Disagree 
Disagree 

Slightly  Disagree 
Neutral 

Slightly  Agree 
Strongly  Agree 


19.6 

53.0 

16.1 


TOTAL 


100.0 


Mean  2.286 

Valid  Cases  168 


Median  2.000  Std  Dev 

Missing  Cases  0 


1.117 


Table  #42 

C42  Pilots-in-command  should  employ  the  same  style  of  management  in  all 
situations  and  with  all  crewmembers. 


Strongly  Disagree 
Disagree 

Slightly  Disagree 
Neutral 

Slightly  Agree 
Agree 

Strongly  Agree 


Value  Frequency  Percent 

1  34  20.2 

2  65  38.7 

3  32  19.0 


TOTAL 


100.0 


Mean  2.720 

Valid  Cases  168 


Median  2.000  Std  Dev 

Missing  Cases  0 


1.582 
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Table  #43 


r 


C43  Pilot-in-command  instructions  to  other  crewmembers  should  be 
general  and  non-specific  so  that  each  individual  can  practice  self¬ 
management  and  can  develop  individual  skills. 


Strongly  Disagree 
Disagree 

Slightly  Disagree 
Neutral 

Slightly  Agree 
Agree 

Strongly  Agree 


Mean  2.833 

Valid  Cases  168 


Value 

Frequency 

Percent 

1 

23 

13.7 

2 

67 

39 . 9 

3 

34 

20.2 

4 

17 

10.1 

5 

16 

9.5 

6 

8 

4.8 

7 

3 

1.8 

TOTAL  168 

Median  2.000 

Missing  Cases  0 


100.0 

Std  Dev 


1.459 


Table  #44 

C44  A  relaxed  attitude  is  essential  to  maintaining  a  cooperative  and 
harmonious  cockpit. 


Disagree 

Slightly  Disagree 
Neutral 

Slightly  Agree 
Agree 

Strongly  Agree 


Mean  5.286 

Valid  Cases  168 


Value  Frequency  Percent 


2 

6 

3.6 

3 

18 

10.7 

4 

12 

7.1 

5 

42 

25.0 

6 

66 

39.3 

7 

24 

14.3 

TOTAL 

168 

100.0 

Median 

6.000 

Std 

Missing  Cases  0 


1.309 
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Table  #45 


C45  Reprimands  are  more  effective  than  discussions  in  eliminating  a 
poor  flying  habit  in  a  crewmember. 

Value  Frequency  Percent 

Strongly  Disagree 
Disagree 

Slightly  Disagree 
Neutral 

Slightly  Agree 
Agree 


Mean  1.946  Median  2.000  Std  Dev  .986 

Valid  Cases  168  Missing  Cases  0 


1  59  35.1 

2  78  46.4 

3  18  10.7 

4  9  5.4 

5  2  1.2 

6  2  1.2 

TOTAL  168  100.0 
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ACE  Checklist  Frequency  Tables 
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APPENDIX  C 


ACE  CHECKLIST  FREQUENCY  TABLES 


Table  #1 


Thorough  pre— flight  mission  plan  developed 

Value  Frequency  Percent 


Very  Poor 

1 

1 

5.0 

Poor 

2 

3 

15.0 

Borderline/Marginal 

3 

3 

15.0 

Fully  Acceptable 

4 

5 

25.0 

Good 

5 

5 

25.0 

Very  Good 

6 

2 

10.0 

Superior 

7 

TOTAL 

1 

20 

5 . 0 

100.0 

Mean  4.000 

Valid  Cases  20 

Std  Err 
Missing  Cases 

.348 

0 

Std 

Table  #2 


A2  Statements/directives  clear,  timely,  relevant 


Value 

Frequency 

Percent 

Very  Poor 

1 

2 

10.0 

Poor 

2 

5 

25.0 

Border 1 ine /Marginal 

3 

7 

35.0 

Fully  Acceptable 

4 

5 

25.0 

Good 

5 

1 

5 . 0 

TOTAL 

20 

100.0 

Mean  2.900 

Std  Err 

.240 

Std 

Valid  Cases  20  Missing  Cases  0 


1.556 


1.071 
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Table  #3 


A3  Inquiry/questioning  practiced 


Poor 

Borderline/Marginal 
Fully  Acceptable 
Good 


Mean  3.150 

Valid  Cases  20 


Value  Frequency 


2  7 

3  5 

4  6 

5  2 

TOTAL  20 

Std  Err  .233 

Missing  Cases  0 


Percent 

35.0 

25.0 

30.0 

10.0 

100.0 

Std  Dev  1.040 


Table  #4 

A4  Advocacy/ assertion  practiced 


Value  Frequency  Percent 


Very  Poor 

1 

1 

5.0 

Poor 

2 

4 

20.0 

Borderline/Marginal 

3 

8 

40.0 

Fully  Acceptable 

4 

6 

30.0 

Good 

5 

TOTAL 

1 

20 

5.0 

100.0 

Mean  3.100 

Valid  Cases  20 

Std  Err 
Missing  Cases 

.216 

0 

Std 

Table  #5 

A5  Decisions  communicated  and  acknowledged 


Value 

Frequency 

Percent 

Poor 

2 

5 

25.0 

Borderline/Marginal 

3 

8 

40.0 

Fully  Acceptable 

4 

6 

30.0 

Good 

5 

1 

5.0 

TOTAL 

20 

100.0 

Mean  3.150 

Std  Err 

.196 

Std 

Valid  Cases  20  Missing  Cases  0 


.875 
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Table  #6 


A6  Actions  communicated  and  acknowledged 

Value  Frequency  Percent 


Poor 

Borderline/Marginal 
Fully  Acceptable 
Good 


Mean  3.200 

Valid  Cases  20 


2  4 

3  9 

4  6 

5  1 

TOTAL  20 

Std  Err  .186 

Missing  Cases  0 


20.0 

45.0 

30.0 

5.0 

100.0 

Std  Dev  .834 


Table  #7 

A7  Crew  self-critique  of  decisions  and  actions 


Very  Poor 
Poor 

Borderline/Marginal 
Fully  Acceptable 
Very  Good 


Mean  3.500 

Valid  Cases  20 


Value  Frequency 


1  1 

2  1 

3  7 

4  10 

6  1 

TOTAL  20 

Std  Err  .224 

Missing  Cases  0 


Percent 

5.0 

5.0 

35.0 

50.0 

5.0 

100.0 

Std  Dev  1.000 


Table  #8 

A8  Crewmember  actions  mutually  cross  monitored 


Very  Poor 
Poor 

Borderline/Marginal 
Fully  Acceptable 
Very  Good 


Mean  3.050 

Valid  Cases  20 


Value  Frequency 


1  1 

2  8 

3  2 

4  8 

6  1 

TOTAL  20 

Std  Err  .276 

Missing  Cases  0 


Percent 

5.0 

40.0 

10.0 

40.0 

5.0 

100.0 

Std  Dev  1.234 
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Table  #9 

Interpersonal  relationships/group  climate 


Borderline/Marginal 
Fully  Acceptable 
Very  Good 


Mean  3.950 

Valid  Cases  20 


Value 

Frequency 

Percent 

3 

3 

15.0 

4 

16 

80.0 

6 

1 

5.0 

TOTAL 

20 

100.0 

Std  Err  .135 

Missing  Cases  0 

Std 

.  605 


Table  #10 

Aircraft,  personnel,  and  mission  status 


Poor 

Borderline/Marginal 
Fully  Acceptable 
Good 


Mean  3.400 

Valid  Cases  20 


Value  Frequency 


2  3 

3  8 

4  7 

5  2 

TOTAL  20 

Std  Err  .197 

Missing  Cases  0 


Percent 

15.0 

40.0 

35.0 

10.0 

100.0 

Std  Dev  .883 


All 


Table  #li 

Distractions  avoided  or  prioritized 


Value  Frequency  Percent 


Very  Poor 
Poor 

Borderline/Marginal 
Fully  Acceptable 
Good 


Mean  3.200 

Valid  Cases  20 


1  1 

2  2 

3  10 

4  6 

5  1 

TOTAL  2  0 

Std  Err  .200 

Missing  Cases  0 


5.0 

10.0 

50.0 

30.0 

5.0 

100.0 

Std  Dev  .894 
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Table  #12 


A12 


Workload  effectively  distributed/redistributed 


Very  Poor 
Poor 

Borderline/Marginal 
Fully  Acceptable 
Good 


Mean  3.550 

Valid  Cases  20 


Value 

Frequency 

Percent 

1 

1 

5.0 

2 

1 

5.0 

3 

5 

25.0 

4 

12 

60.0 

5 

1 

5.0 

TOTAL 

20 

100.0 

Std  Err 

.198 

Std 

Missing  Cases  0 


A13 


Table  #13 

Support  information/actions  sought  from  crew 


Very  Poor 
Poor 

Borderline/Marginal 
Fully  Acceptable 
Good 


Mean  3.200 

Valid  Cases  20 


Value  Frequency  Percent 


1 

2 

3 

4 

5 

TOTAL 

Std  Err 
Missing  Cases 


2  10.0 

3  15.0 

6  30.0 

7  35.0 

2  10.0 

20  100.0 

258  Std  Dev 

0 


Table  #14 


A14 


Support  information/actions  offered  by  crew 


Very  Poor 
Poor 

Borderline/Marginal 
Fully  Acceptable 
Good 


Value  Frequency 

Percent 

1  2 

10.0 

2  5 

25.0 

3  5 

25.0 

4  5 

25.0 

5  3 

15.0 

TOTAL  20 

100.0 

Std  Err  .280 

Missing  Cases  0 

Std 

.887 


1.152 


Mean 

Valid  Cases 


3.100 

20 
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1.252 
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Table  #15 

Overall  technical  proficiency 


Very  Low 


Very  High 


Mean  3.250 

Valid  Cases  20 


Value  Frequency 


1  1 

2  2 

3  9 

4  7 

5  1 

TOTAL  2  0 

Std  Err  .204 

Missing  Cases  0 


Percent 

5.0 

10.0 

45.0 

35.0 

5.0 

100.0 

Std  Dev  .910 


A16 


Table  #16 

Overall  crew  effectiveness 


Very  Low 


Very  High 


Mean  3.300 

Valid  Cases  20 


Value  Frequency 


1  2 

2  1 

3  9 

4  5 

5  3 

TOTAL  2  0 

Std  Err  .252 

Missing  Cases  0 


A17 


Table  #17 


Overall  workload 


Very  Low 


Very  High 


Mean 

Valid  Cases 


Value  Frequency 


1  1 

2  1 

3  1 

4  15 

5  2 

TOTAL  20 


Std  Err 
Missing  Cases 


Percent 

10.0 

5.0 

45.0 

25.0 

15.0 

100.0 


Std  Dev 


Percent 


5.0 

5.0 

5.0 

75.0 

10.0 

100.0 


1.129 


.894 


3 . 800 
20 
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200 

0 


Std  Dev 


Table  #18 


i 


A18  Management  of  abnormal  or  emergency  situations 


Very  Poor 
Poor 

Borderline /Margina 1 
Fully  Acceptable 
Good 


Mean  3.050 

Valid  Cases  20 


Value  Frequency  Percent 


1  1  5.0 

2  5  25.0 

3  7  35.0 

4  6  30.0 

5  1  5.0 


TOTAL 


20  100.0 


Std  Err  .223 

Missing  Cases  0 


Std  Dev 


Table  #19 


A19  Conflict  resolution 


Value  Frequency  Percent 


Poor 

2 

5 

25.0 

Borderline/Marginal 

3 

6 

30.0 

Fully  Acceptable 

4 

7 

35.0 

Good 

5 

1 

5.0 

Very  Good 

6 

TOTAL 

1 

20 

5.0 

100.0 

Mean  3.350 

Valid  Cases  20 

Std  Err 
Missing  Cases 

.244 

0 

Std 

.999 


1.089 
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Revised  ATM  Tasks,  Modified 
Frequency  Tables 


Gradeslips 
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APPENDIX  D 
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ATM  FREQUENCY  TABLES 


Table  #1 


BIGRADE  Overall  grade  for  flight 


Value  Frequency  Percent 


1  4  20.0 

2  8  40,0 

3  8  40.0 


Mean  2.200 

Valid  Cases  20 


TOTAL  20  100.0 


Median  2.000  Std  Dev 

Missing  Cases  0 


Table  #2 


TlOOl  Task  1001  VFR  Flight  Planning 


Value 

Frequency 

Percent 

u 

1 

3 

15.0 

c 

2 

3 

15.0 

B 

3 

7 

35.0 

A 

4 

7 

35.0 

TOTAL 

20 

100.0 

Mean  2.900 

Median 

3.000 

Std 

Valid  Cases  20 

Missing  Cases  0 

.768 


1.071 


D-3 


T1003 


Task  1003 


Table  #3 
DD  Form  365-4 


Value  Frequency  Percent 


Mean 

Valid  Cases 


2.900 

20 


TOTAL 


3  15.0 

3  15.0 

7  35.0 

7  35.0 

20  100.0 


Median  3.000  std  Dev 

Missing  Cases  0 


1.071 


T1004 


Task  1004 


Mean  2.417 

Valid  Cases  12 


Table  #4 
DD  Form  5701-R 

Value  Frequency  Percent 


total  20 

Median  2.000 

Missing  Cases  8 


5.0 

35.0 

10.0 

10.0 

40.0 

100.0 


Std  Dev 


.900 


T1007 


Task  1007 


2.450 

Valid  Cases  20 


Table  #5 

Engine  Start,  Runup  and  Before  Takeoff  Checks 
Value  Frequency  Percent 

^  5  25.0 

2  3  15.0 

2  10  50.0 

4  2  10.0 


TOTAL  20 

Median  3.000 

Missing  Cases  0 


100.0 


Std  Dev 
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Table  #6 


Ground  Taxi 

Value  Frequency  Percent 

3  1  5.0 

19  95.0 

TOTAL  20  100.0 

Median  3.000 

Missing  Cases  19 

Table  #7 

Hover  Power  Check 

Value  Frequency  Percent 


T1015  Task  1015 


B 


Mean  3.000 

Valid  Cases  1 


T1016  Task  1016 


U 

C 

B 

A 


Mean  1.900 

Valid  Cases  20 


T1017  Task  1017 


c 

B 


Mean  2.800 

Valid  Cases  20 


1  10 

2  4 

3  4 

4  2 

TOTAL  2  0 

Median  1.500 

Missing  Cases  0 


Table  #8 
Hovering  Flight 

Value  Frequency 


2  4 
3  16 

TOTAL  2  0 

Median  3.000 

Missing  Cases  0 


50.0 

20.0 

20.0 

10.0 

100.0 

Std  Dev  1.071 


Percent 

20.0 

80.0 

100.0 

Std  Dev  .410 
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Table  #9 


T1018  Task  1018 


B 


Mean  3.000 

Valid  Cases  1 


Normal  Takeoff 


Value 

Frequency 

Percent 

3 

1 

5.0 

• 

19 

95.0 

TOTAL 

20 

100.0 

Median 

3.000 

Missing 

Cases  19 

T1023  Task  1023 


U 

C 

B 

A 


Mean  1.421 

Valid  Cases  19 


Table  #10 

Fuel  Management  Procedures 
Value  Frequency  Percent 


1  15 

2  1 

3  2 

4  1 

1 

TOTAL  20 

Median  1.000 

Missing  Cases  1 


75.0 

5.0 

10.0 

5.0 

5.0 

100.0 

Std  Dev 


T1026  Task  1026 


U 

C 

B 

A 


Mean  2.600 

Valid  Cases  20 


Table  #11 


Doppler  Navigation 


Value  Frequency  Percent 


1  3  15.0 

2  5  25.0 

3  9  45.0 

4  3  15.0 


TOTAL  20  100.0 


Median  3.000  Std  Dev 

Missing  Cases  0 


Table  #12 


i 


T1027  Task  1027 


U 

C 

B 


Mean  1.800 

Valid  Cases  20 


Before  Landing  Check 

Value  Frequency  Percent 


1  9  45.0 

2  6  30.0 

3  5  25.0 

TOTAL  20  100.0 


Median  2.000  Std  Dev 

Missing  Cases  0 


T1028  Task  1028 


C 

B 


Mean  2.667 

Valid  Cases  3 


Table  #13 
VMC  Approach 


Value 

2 

3 

• 

TOTAL 

Median 

Missing 


Frequency 

1 

2 

17 

20 

3 . 000 
17 


Percent 

5.0 

10.0 

85.0 

100.0 

Std  Dev 


Cases 


T1031  Task  1031 


C 

B  . 


Mean  2.700 

Valid  Cases  20 


Table  #14 


Confined  Area  Operations 


Value 

Frequency 

Percent 

2 

6 

30.0 

3 

14 

70.0 

TOTAL 

20 

100.0 

Median  3.000 

Missing  Cases  0 

Std 

.834 


.577 


.470 


Table  #15 


T1036  Task  1036 


U 

C 

B 


Mean  2.250 

Valid  Cases  4 


T1063  Task  1063 


U 

C 

B 


Mean  1.900 

Valid  Cases  20 


T1068  Task  1068 


U 

C 

B 


Mean  1.950 

Valid  Cases  20 


Hover  OGE  Check 

Value  Frequency 


1  1 

2  1 

3  2 

16 

TOTAL  20 

Median  2.500 

Missing  Cases  16 


Table  #16 


Median  2.000 

Missing  Cases  0 


Table  #17 

Emergency  Procedures 


Median  2.000 

Missing  Cases  0 


Percent 

5.0 

5.0 

10.0 

80.0 

100.0 

Std  Dev  .957 


Std  Dev  .852 


Std  Dev  .887 


Stabilator  Malfunction  Procedures 
Value  Frequency  Percent 


1 

2 

3 


8 

6 

6 


40.0 

30.0 

30.0 


TOTAL 


20 


100.0 


Value  Frequency  Percent 

1  8  40.0 

2  5  25.0 

3  7  35.0 

TOTAL  20  100.0 


D-8 


T1071 


Task  1071 


Table  #18 

Aircrew  Coordination 


Percent 


U 

c 

B 

A 


Mean  2.200 

Valid  Cases  20 


T1076  Task  1076 


C 

B 


Mean  2.800 

Valid  Cases  15 


Value  Frequency 

1 
2 

3 

4 

TOTAL 

Median  2.000  Std  Dev 

Missing  Cases  0 


6  30.0 

5  25.0 

8  40.0 

1  5.0 

20  100.0 


Table  #19 
Radio  Navigation 

Value  Frequency  Percent 


2  3  15.0 

3  12  60.0 

5  25.0 


TOTAL  20  100.0 

Median  3.000  Std  Dev 

Missing  Cases  5 


.951 


.414 


Table  #20 

Radio  CoTnmunication  Procedures 
Value  Frequency  Percent 


T1079  Task  1079 


C 

B 


Mean  2.750 

Valid  Cases  20 


2  5 

3  15 

TOTAL  20 

Median  3.000 

Missing  Cases  0 


25.0 

75.0 

100.0 

Std  Dev  .444 


D-9 


T1081  Task  1081 


U 

C 

B 

A 


Mean  2.444 

Valid  Cases  18 


T1083  Task  1083 


U 

C 

B 

A 


Mean  2.667 

Valid  Cases  18 


T1095  Task  1095 


U 

C 

B 


Mean  2.050 

Valid  Cases  20 


Table  #21 


Nonprecision  Approach 


Value 

Frequency 

Percent 

1 

4 

20.0 

2 

3 

15.0 

3 

10 

50.0 

4 

1 

5.0 

• 

2 

10.0 

TOTAL 

20 

100.0 

Median 

3.000 

Std  Dev 

Missing  Cases  2 

Table  #22 

VHIRP 

Value 

Frequency 

Percent 

1 

2 

10.0 

2 

7 

35.0 

3 

4 

20.0 

4 

5 

25.0 

• 

2 

10.0 

TOTAL 

20 

100.0 

Median 

2.500 

Std  Dev 

Missing  Cases  2 

Table  #23 

Aircraft  , 

Survivability  Equipment 

Value 

Frequency 

Percent 

1 

8 

40.0 

2 

3 

15.0 

3 

9 

45.0 

TOTAL 

20 

100.0 

Median 

2.000 

Std  Dev 

Missing  Cases  0 


.922 


1.029 


.  945 


D-10 


>  w  o  c; 


* 


T1098  Task  1098 


U 

C 

B 


Mean  1.727 

Valid  Cases  11 


T1099  Task  1099 


U 

C 

B 

A 


Mean  2.300 

Valid  Cases  20 


T2008  Task  2008 


2.700 

20 


Mean 

Valid  Cases 


Table  #24 

After  Landing  Tasks 
Value  Frequency 


1  6 

2  2 

3  3 

9 

TOTAL  2  0 

Median  *  1.000 

Missing  Cases  9 


Table  #25 

Mark  XII  IFF  System 
Value  Frequency 


1  8 

2  1 

3  8 

4  3 

TOTAL  2  0 

Median  3.000 

Missing  Cases  0 


Percent 

30.0 

10.0 

15.0 

45.0 

100.0 

Std  Dev 


Percent 

40.0 

5.0 

40.0 

15.0 

100.0 

Std  Dev 


Table  #26 
Evasive  Maneuvers 


Value 

Frequency 

Percent 

1 

2 

10.0 

2 

4 

20.0 

3 

12 

60.0 

4 

2 

10.0 

TOTAL 

20 

100.0 

Median  3.000 

Missing  Cases  0 

Std 

.905 


1.174 


.801 


D-11 


» 


T2009  Task  2009 


U 

C 

B 

A 


Mean  2.600 

Valid  Cases  20 


T2016  Task  2016 


U 

C 

B 


Mean  2.400 

Valid  Cases  20 


T2081  Task  2081 


U 

C 

B 

A 


Mean  2.450 

Valid  Cases  20 


Table  #27 


Multiaircraft  Operations 


Value 

Frequency 

Percent 

1 

2 

10.0 

2 

5 

25.0 

3 

12 

60.0 

4 

1 

5.0 

TOTAL 

20 

100.0 

Median 

3.000 

Std 

Missing  Cases  0 

Table  #28 

Perform  External  Load  Operations 


Value 

Frequency 

Percent 

1 

4 

20.0 

2 

4 

20.0 

3 

12 

60.0 

TOTAL 

20 

100.0 

Median 

3.000 

Std 

Missing  Cases  0 


Table  #29 

Perform  Terrain  Flight 


Value 

Frequency 

Percent 

1 

4 

20.0 

2 

5 

25.0 

3 

9 

45.0 

4 

2 

10.0 

TOTAL 

20 

100.0 

Median 

3.000 

Std 

Missing  Cases 


.754 


.821 


.945 


D-12 


>  CD  n  <3 


Table  #30 


T2084  Task  2084 


Mean  2.600 

Valid  Cases  20 


Perform  Terrain  Flight  Approach 


Value 

Frequency 

Percent 

1 

1 

5.0 

2 

7 

35.0 

3 

11 

55.0 

4 

1 

5.0 

TOTAL 

20 

100.0 

Median 
Missing  ' 

3.000 
Cases  0 

Std 

.681 


D-13 


Bivariate 

"Logical 


APPENDIX  E 


Correlation  Tables  for  Army  CMAQ 
"  Subscales  and  Other  Variables 
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appendix  f 


Bivariate 

"Factor” 


Correlation  Tables  for  Army  CMAQ 
Subscales  and  Other  Variables 
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